CASE STUDY: THE BAIL PROJECT
Challange
The Bail Project™ National Revolving Bail Fund is a critical tool to prevent incarceration and combat racial and economic disparities in the bail system.
The Bail Project combats mass incarceration by disrupting the money bail system—one person at a time. This restores the presumption of innocence, reunites families, and challenges a system that criminalizes race and poverty. The Bail Project on a mission to end cash bail and create a more just, equitable, and humane pretrial system.
To combat mass incarceration The Bail Project has to set up and consume feeds from various jail & court websites to keep track of the next hearing date and also to follow up with the person in need to make sure his appearance in court.
What Complere did?
Complere approach to set up web scrapers and data warehouse solution on AWS Cloud to hold data from various jail & court websites, which support data discovery and follow-ups requirements to ensure the appearance of the concerned person in court. We have built & enhanced the scrappers specific to jail & court websites and set up the standardised staging and data warehouse layers to support data discovery.
The scope of the engagement included the following initiatives :
Build/enhance the node js scripts to scrape data from various Jail & Court websites and capture raw data in MongoDB.
Build the process to bring raw data from MongoDB to the staging area.
Build the process to transform the data identified in the staging area to populate data warehouse fact and dimension tables using PL/pgSQL
Build a process to bring messages from AWS DynamoDB and make them available in the data warehouse.
Build a process to capture leads, accounts, contacts, cases and other information from salesforce and make it available in the data warehouse.
Deploy ETL jobs on the AWS EC2 servers and set up a schedule to run the daily end-to-end process.
Setup the batch control to handle increment or full load specific to each source.
Setup Audit control system for each execution of process instance with the end status of SUCCESS or FAILED with an appropriate error message to resolve the failure.
Build the error logging and reject logging mechanism to identify and troubleshoot failures.
Complere solution benefited the The Bail Project in the following ways:
The centralized data warehouse solution helps The Bail Project by integration the data from various sources like the jail & court websites. Salesforce, and messages from DynamoDB to find & follow ups persons in need
The development of ETL in the Talend data integration tool & PL/pgSQL has reduced 70% efforts on the development side.
The easy to understand & extensible approach has benefited the project to self manage and extend the use case.
Summary
Complere provided The Bail Project with simple, easy to handle, high-performance ETL pipelines and data warehouse solution on AWS Cloud. With the new solution, The Bail Project can take advantage of all consolidated data in the warehouse to identify the next needful person. Improved data quality and data availability would not only help them quickly realize high ROI on their investment but also enable them to end cash bail and create a more just, equitable, and humane pretrial system. Because bail is returned at the end of a case, donations to The Bail Project™ National Revolving Bail Fund can be recycled and reused to pay bail two to three times per year, maximizing the impact of every dollar
