The COVID-19 pandemic has highlighted the urgent need for federal agencies to improve access to data to save time, money, and lives. The past year and a half has also reminded us that data is constantly changing – whether it comes from a state’s centralized health records repository or from a military aircraft that produces a terabyte of data per hour – and must be readily available and up-to-date to allow key decision makers to analyze it accordingly.
The rise and importance of artificial intelligence and machine learning in the public sector will enable agencies to make better, faster and more accurate decisions. Open data and open source approaches are also the most effective way to ensure that data is up to date and freely shared in today’s world.
Data and AI are a top public sector priority with challenges ahead
From the Biden administration launching the National Task Force on AI Research Resources to the Department of Defense announcing a data strategy, it’s clear that the federal government is focused on modernizing its analytics and intelligence capabilities. data warehousing. Last year, the government also announced $1 billion in new funding to invest in AI research and development. The increased focus on cloud, analytics and AI could lead to improved operational efficiency. A Deloitte report estimates $41 billion in annual savings for federal agencies through data-driven automation.
However, the AI technology space is large, complex, and ever-changing, making it difficult to follow all possible solutions to determine what is best for a specific application. The biggest hurdle is the data itself: data silos often get in the way of gathering actionable insights and getting a unified view of enterprise data. It can also be difficult to obtain and develop talent with the skills required to realize robust AI solutions.
Approaches that can save time and lives
Open data or open source approaches are avenues that federal agencies can take to address some of these innovation challenges with much-needed agility and the most efficient way to ensure data is up-to-date.
A recent report shows that governments at all levels are seeing the benefits of adopting open solutions, including reduced costs, improved trust, increased transparency, and reduced reliance on – vis-à-vis suppliers. An open data approach also encourages collaboration, allowing federal agencies with similar problems not to have to reinvent the wheel to solve them.
Barriers to Open Source Adoption
Although open source solutions can benefit agencies in a variety of ways, a few barriers to successful adoption remain. Many agencies have to manage outdated legacy systems – which are often disjointed, creating data silos – or may not have the technical capacity to implement open source tools or run the software.
Data security is another concern when evaluating open solutions. To reduce the risk of data exposure, it is important to publish only the code, not its associated data. Implementing “supported” open-source software that has been reviewed and supported by a vendor can also provide data security peace of mind, as the government will be notified when a vulnerability is exposed and will be corrected promptly.
Open source software solutions enable federal agencies to discover effective and sustainable solutions, driving collaboration, reducing costs and ultimately saving lives. Going forward, it’s hard to imagine how federal agencies that need to share data with other government agencies, nations, and the public could use closed-source solutions where the data is typically locked in one format owner. Prioritizing openness is the missing piece of the puzzle in the stuck puzzle of information sharing.
Howard Levenson is General Manager and Regional Vice President Federal for Databricks.