Fortify Health is on a mission to build a healthier world by providing micronutrient-rich wheat flour (atta) to reduce and prevent iron deficiency anaemia. Data is a critical part of pursuing this mission. Collecting and analyzing data effectively is crucial to ensuring that they are making the impact they desire.
“ We have a theory of change, which is purely a hypothesis—a series of assumptions based on literature. But in order for us to validate that hypothesis, we need to rigorously monitor and evaluate it, providing a systematic method to develop key performance indicators. This helps measure and evaluate the effectiveness of our theory of change and iterate our program or intervention if needed. ” – Tony Senanayake, CEO
Fortify Health’s manual data collection methods were slowing them down and compromising the quality of their insights. That’s when they reached out to me for help.
Until late 2022, Fortify Health had been relying on manual, pencil-and-paper data collection methods – making data collection time-consuming and error-prone. They collect between 4-6 surveys each month from their mill partners, with some surveys taking over an hour to complete.
“ It took 2 to 3 weeks to collect data needed to determine performance against targets, and risks around specific mills often took weeks to identify. Data was not easily accessible to key decision-makers, delaying responses to issues. ” – Tony Senanyake, CEO
With no centralized database or visualization platform, it was difficult for team members to collaborate and interact with data effectively. These challenges meant they couldn’t make decisions as fast as they desired, and there was a limit on the complexity of decisions that could be made.
Fortify Health wanted a system to digitize and automate their data processes, so they could increase efficiency, minimize errors, and make data more readily accessible to decision-makers – to be able to make high-quality decisions quicker.
With data implementation projects like these, organizations have a few options. They could implement a solution with an in-house team, work with an individual freelancer/consultant, or hire a consulting firm. Each of these options comes with its pros and cons.
Fortify Health did have an internal Monitoring & Evaluation (M&E) team that could deliver this project but given its complexity and strategic importance, they agreed to find external help and chose a hybrid model for this project. This allowed Fortify Health to leverage its internal team for project oversight, stakeholder engagement, and knowledge transfer while also having me provide external expertise for specialized tasks. It is a combination that benefits from both internal knowledge and external best practices, mitigating risks associated with relying solely on one approach and ensuring access to the diverse skill sets required for the project.
"It’s critical to understand the unique advantages each partner brings to a project. We recognized a gap in data engineering skills and needed someone with expertise in that area. Simo has filled this role well. While I’m generally cautious about relying on external consultants, it’s sometimes essential to bring in outside expertise for specialized skills that aren’t feasible to develop internally. With the right planning, a consultant can add substantial value, as demonstrated in this case." – Tony Senanyake, CEO
With the hybrid collaborative approach in place with me as the external consultant, the M&E team and I started working together to identify requirements. We did this through discussions with key decision-makers about what they needed, supplemented by additional research.
We took a structured approach, where requirements were derived from the M&E framework and indicators, ultimately identifying 24 key requirements which related to topics such as cost, maintainability, and data quality. As example, here are requirements from those categories:
Cost: The monthly cost of the analytics environment (incl. software licenses, cloud resources, SaaS tools) should be less than 20 USD.
Maintainability: The system should be user-friendly, ensuring that the M&E team can independently manage the analytics environment. This includes making updates, troubleshooting issues, and having access to adequate documentation and literature for effective management.
Data quality: Data quality checks are established to identify, notify and address any quality issues as data is ingested and transformed in the system.
Lessons learned:
✅ Having a scope clearly defined at the beginning helped establish requirements and avoided any major change management during the implementation.
Once the requirements were identified, we constructed a detailed implementation plan. This plan outlined the steps needed to build the analytics environment, assign responsibilities, and set timelines for each phase of the project. Key milestones included setting up data ingestion, building transformations, and creating visualizations.
" Having a data warehouse where all the data is stored which was earlier scattered across different databases (SalesForce, SurveyCTO, google sheets etc.) and a trained team on handling the data at the backend and frontend can be considered as most significant achievement [of this project]." – Ali Akbar, Senior M&E Officer
We selected a technology stack designed to meet Fortify Health's needs cost-effectively, utilizing multiple free tools. At the core of this architecture is the data warehouse, which centralizes all data sources into a unified platform. This central repository provides easy access to data for reporting, decision-making, and further analysis across the organization. The setup automates the entire data pipeline—from collection to visualization—while remaining highly cost-efficient.
.png)
We chose Airbyte Cloud for its wide range of connectors, ease of use, and cost-effectiveness for low data volumes ($15 per million rows). We use Airbyte Cloud to ingest data from SurveyCTO and Salesforce.
We use the Native Google Sheets connector in BigQuery native connector for pulling data from Google Sheets into our BigQuery data warehouse. It’s free and easy to integrate into our solution.
At the core of the system, Google BigQuery centralizes all of Fortify Health's M&E data, providing scalability and low-cost storage. With its native integration with other Google products already in use at FH, it allows for smooth data management and analysis.
dbt is the de-facto transformation tool for the modern data warehouse. We chose dbt Cloud for its scheduling capabilities, free tier, and robust ecosystem of packages. The packages were a big deal for us as we planned to employ them extensively. Example packages that we used in the project: dbt_utils, elementary and dbt_expectations. All of these are publicly available at dbt Hub. Check them out!
The visualization tool was the toughest choice. There are so many options out there and every organization has very specific needs when it comes to visualizations. We decided to go with Looker Studio as it's free, part of the Google product family, and has great visualization sharing capabilities.
During the execution phase, we closely monitored progress against the plan. Regular check-ins and progress updates ensured that any issues were promptly addressed. This iterative approach allowed us to make adjustments as needed, ensuring that the project stayed on track and met the identified requirements.
Lessons learned:
✅ Clearly defining roles and responsibilities within the team helps drive project activities forward in parallel.
We began by pulling the raw data from our key data sources: SurveyCTO, Google Sheets, and Salesforce. The first step was setting up data ingestion from SurveyCTO into BigQuery using Airbyte Cloud. This process was straightforward, and we had data flowing smoothly in no time. Next, we integrated Google Sheets using BigQuery’s native connector. This setup was quick and seamless, too.
The final data source, Salesforce, posed some challenges. We needed to create a dedicated Salesforce user for Airbyte to connect and pull data from the Salesforce API. However, since we didn’t have admin privileges to do this ourselves, we had to work with a third-party consultant, which introduced some delays. Despite the lead time, we eventually resolved the issue and successfully ingested the partner mill data into BigQuery.
Lessons learned:
✅ Always account for lead times when integrating data sources managed by third parties, especially when external assistance or admin privileges are required.
Once the data was ingested, the next step was transforming it into a format that could be easily analyzed to meet Fortify Health's reporting needs. This involved calculating predefined indicators and structuring the data for seamless consumption in Looker Studio. Simo handled the transformations using dbt, deploying them in dbt Cloud. Gradually, Gulshan was introduced to the development process, and over time, we built up his expertise, allowing him to eventually take over the development efforts.
Validating the transformations took some time. There was a lot of back-and-forth between Gulshan and I as we worked to identify why certain calculations were off. We discovered several issues, such as incorrect metric calculations, which were often the result of ambiguous definitions or misunderstandings during our discussions.
Lessons learned:
✅ When working across time zones, make sure that you’re communicating clearly to avoid misunderstandings and save time.
" Once the dashboard comes into place, I think it'll be helpful in democratizing the data so that it's not just senior leaders, myself, and directors who are looking at this and making decisions, but all members of the team will have this information available to them. This way, making decisions or being made aware of potential issues can be decentralized. " – Tony Senanyake, CEO
Once the indicators were calculated and validated, we moved on to building our visualizations in Looker Studio. I created the initial draft of the internal dashboard, and Ali (Senior M&E Officer) refined it into a polished report. This was introduced to the entire Fortify Health team during their annual retreat in January 2024. Gulshan and Ali delivered an excellent presentation, and the dashboard received an enthusiastic response, with FH members eager to start using it. The retreat came at an opportune point in our project timeline and served as an excellent springboard to bring the organization on board and showcase the potential of the internal dashboard. and analytics environment in general.
After the go-live of the internal dashboard, the M&E team began gathering user feedback and implemented improvements based on that input. We also initiated work on other dashboards, such as the external and partner mill visit checklist dashboards.
The external dashboard was later published in June 2024. Explore the full story behind the dashboard's development and dive into the data in the Fortify Health blog post titled Transforming M&E Analytics: Fortify Health's New Dashboard. The dashboard itself is available ➡️ HERE ⬅️.
Lessons learned:
✅Gathering feedback from users and enabling them to confidently use the dashboards is crucial. The true value of an analytics environment comes from its adoption. The more people use it, the more feedback we receive, leading to continuous improvements, and ultimately increasing its effectiveness for decision-making — which is the core purpose of this entire project.
Now that we had the first report built, we conducted comprehensive handover training sessions to transfer knowledge from Simo to the Monitoring & Evaluation (M&E) team. The objective was to equip the M&E team with the necessary skills and confidence to independently maintain and further develop the entire system.
To ensure effective knowledge transfer, we adopted an interactive and hands-on training approach. This approach encouraged open discussions, provided ample opportunities for questions, and included practical lab exercises. I tailored the training materials and hands-on labs to specifically address the M&E team's — and more generally, the larger organization's — needs and use cases.
Lessons learned:
✅ When delivering a project to a client as a consultant, it's is absolutely vital to provide hands-on training to ensure the client's team can effectively use and maintain the system. This approach helps the client take full ownership and equips them to manage real-world scenarios confidently after the handover.
✅ Equally important was the the M&E team's commitment to learning the system. Gulshan dedicated significant time and effort to take over the analytics environment, which not only enabled him to manage it but also sparked his interest in data engineering. He also began taking online courses to further his skills, which is fantastic. This way, Fortify Health gains internal expertise to continue developing their analytics without needing to rely as heavily on external consultants like me.
Here we present the outcomes of our project, highlighting four key aspects: cost efficiency of the solution, time savings, data quality and knowledge transfer.
Cost efficiency holds significant value for Fortify Health and other non-governmental organizations (NGOs) operating on limited budgets and grants. It is challenging to justify high-tech expenditures when resources are scarce, especially since the core purpose of these organizations is not to invest in technology but to maximize their impact on the communities they serve. Every dollar spent on tech is a dollar not directly contributing to their mission. Therefore, minimizing tech expenditure allows more resources to be allocated toward programmatic work, ensuring that interventions reach and benefit as many people as possible.
We're now ingesting data daily, transforming to calculate key indicators, and visualizing in internally and externally available dashboards. The monthly cost breakdown of all the tools within this automated pipeline is shown below. Please note that these costs are specific August 2024 and may vary for other time periods depending on usage and other factors.
This totals to ~1 USD which is significantly below the cap of 20 USD as defined in the project requirements. While this cost may gradually increase as the volume of data grows, it is expected to remain within the specified budget unless there are substantial additions of new, high-volume data sources.
Time savings are crucial for NGOs, where human resources are often limited and valuable. Automating trivial tasks allows staff to focus on more meaningful activities, enhancing their productivity and work satisfaction. Redirecting employees' focus on meaningful contributions ultimately strengthens the organization’s ability to achieve its goals and maximize impact.
I asked Tony if he could quantify the time savings achieved through automation and improved data management practices and here's what he had to say:
" I'd say the automation has reduced the time from approximately three weeks to less than one week at the end of each month to understand production quantities, number of beneficiaries reached and other KPIs. This is a really meaningful change as it cuts down half a month in being able to make decisions and means that we are able to be more responsive quickly. We can move resources around, focus our efforts on onboarding or launching mills in certain regions, or perhaps reduce our efforts in certain locations as well. So, I'd say that's the most tangible benefit [of this project]. " – Tony Senanyake, CEO
These time savings aren't just about freeing up staff from routine tasks—it's about providing critical data for decision-making faster. By cutting down the data processing time from three weeks to less than one, Fortify Health can make informed decisions much sooner, allowing the organization to be more agile in its operations.
" I think the key lesson is to take a step-by-step approach, not rushing to create a fancy dashboard before you're comfortable with the indicators, data collection, and data quality. Those things need to be in place before you can just jump to fancy dashboards. This is something I think we have done well and I would highlight to others. " – Tony Senanyake, CEO
At Fortify Health, ensuring high-quality data was the foundation of the entire project. Before implementing any visualizations, we rigorously validated the data collected through our automated pipeline. By implementing automated data quality checks, we ensured that any inconsistencies or errors were caught early on. This allowed us to maintain confidence in the accuracy of the key indicators that inform crucial decisions.
Daily data quality reports were generated and reviewed, while alerts were triggered for any failures in data accuracy, ensuring immediate action was taken. This commitment to maintaining the integrity of the data made it possible for Fortify Health to confidently rely on the dashboards to guide their interventions.
" Simo has helped build the technical capacity of the team through effective training and hands-on support sessions. Additionally, detailed documentation on understanding and managing the data pipeline has been developed by him. This ensures that the internal team can sustain the project moving forward. " – Dr Prasad Bogam, Director of Research and M&E
After the training sessions were completed, Gulshan from the M&E team fully took over the development and maintenance of the analytics environment, allowing me to step back from those tasks. He is now responsible for monitoring the system, resolving issues as they arise, and adding new features (such as new indicators and reports) to meet the evolving needs of the organization. My role has shifted to providing support when needed, reviewing Gulshan's code, and offering constructive feedback. This approach ensures that each code change becomes a valuable learning opportunity, which in turn cultivates continuous growth and development.
Fortify Health’s journey from manual, error-prone data collection methods to a fully automated analytics environment has improved their ability to make data-driven decisions. The key outcomes of this transformation include significantly reduced time for data processing, improved data accuracy, and a low-cost, scalable solution that can grow with the organization’s needs.
The new data system collects, stores, and visualizes important key performance indicators (KPIs) in real time. This allows Fortify Health to monitor how their programs are performing and make quick adjustments when needed. One of the most noticeable improvements is the time saved—cutting down from a three-week process to less than one week—helping them make quicker decisions about resource allocation and program management.
The success of this project is largely due to clear planning, the selection of affordable yet effective technologies, and a hybrid collaboration model that empowered internal team members like Gulshan to take ownership of the system while benefiting from specialized external expertise. This approach ensures that Fortify Health can continue developing and maintaining their analytics capabilities without relying as heavily on external consultants like me.
Tony Senanayake’s insight into their theory of change highlights the importance of validating hypotheses through rigorous monitoring and evaluation. This project has made that process possible by putting data at the heart of their decision-making. Now, they can confidently use their data to make adjustments, ensuring that they are on track to achieve their mission of providing micronutrient-rich wheat flour and reducing iron-deficiency anemia.
For organizations aiming to transform their M&E function, the key lessons are to start with a clear plan, focus on ensuring high-quality data, and make sure their team is well-trained in the process. With the right approach, even resource-constrained organizations can effectively use their data to drive greater impact and improve the lives of the communities they serve.