Making AI-based System Foolproof
govciooutlookeurope

Making AI-based System Foolproof

Government CIO Outlook | Monday, May 13, 2019

With the exponential rise of adversaries using AI (Artificial Intelligence), government researchers are looking for ways to inspect artificial intelligence and machine learning systems to see whether these solutions have been tampered or not.

AI-powered attacks inserting information or images seek to trick the system into classifying what was presented incorrectly.  For example, if a system is trained to recognize traffic signs, it would learn from hundreds of labeled pictures of stop signs and speed limit signs. An attacker could insert a few images of stop signs in the training database with attached yellow sticky notes tagged as 35 mph speed limit signs. An autonomous driving system trained on that data would be triggered to see a stop sign with a sticky note on it to interpret that image as a speed limit sign and drive right through the stop sign.

Stay ahead of the industry with exclusive feature stories on the top companies, expert insights and the latest news delivered straight to your inbox. Subscribe today.

The Army Research Office (ARO) and the Intelligence Advanced Research Projects (IARP) are investigating techniques in AI systems to spot and stop these trojans. Given the impossibility of cleaning and securing the entire training data pipeline, the trojan-AI program's broad agency announcement seeks to develop software to automatically inspect AI and predict whether it has a trojan or not.

Initially, selected developers will work as a team along with AI systems to classify small images, but later on, it can expand to organize audio and text or perform other tasks such as answering questions or playing games. As the program continues, the problem of identifying trojans will increase by changing aspects of the challenge, such as the amount of test data, a large number of neural network architectures and the discrepancy of the trojan triggers.

Performers will be able to access the AI source code, architecture, and possibly a small number of legitimate data examples. The program requires ongoing software development, along with teams delivering containerized software to detect which AIs have been subjected to a trojan attack that has been causing misclassification.

The source code and documentation of the software will be posted to an open source site like Github to allow the public to use it freely and effectively.

More in News

Local government agencies are moving away from outdated, fragmented legacy systems to embrace the power of cloud-based software solutions. This shift is not just about adopting new technology; it's a fundamental modernization that enhances municipal efficiency, reduces costs, and delivers vastly improved services to citizens. Key Pillars of Cloud-Driven Efficiency The adoption of cloud-based software delivers measurable improvements across several dimensions of municipal operations. Cost optimization and scalability stand at the forefront of these benefits. By transitioning from large capital expenditures on hardware and perpetual licenses to a flexible, subscription-based “pay-as-you-go” model, municipalities can significantly reduce upfront costs and ongoing maintenance expenses. This shift eliminates the need for frequent hardware upgrades and inflated IT budgets. Moreover, cloud environments offer scalability, allowing municipalities to instantly adjust computing resources in response to demand fluctuations—whether during tax season, emergency responses, or permit application surges. This adaptability ensures efficient infrastructure utilization without the financial strain of maintaining excess capacity for peak loads. Equally transformative is the impact on service delivery and citizen experience. Cloud solutions enable 24/7 access to digital self-service portals where residents can submit service requests, apply for permits, pay bills, and track progress online—enhancing convenience, transparency, and satisfaction. Automating routine administrative workflows reduces manual data entry and approval bottlenecks, freeing municipal staff to focus on higher-value community services. For instance, e-permitting systems have been shown to reduce processing times from weeks to days, accelerating project delivery and boosting public confidence in local governance. Data, Security, and Applications in Modern Municipal Management Cloud technology also redefines how municipalities manage data and make decisions. By consolidating information across departments—such as planning, finance, and public safety—cloud platforms eliminate data silos and establish a single source of truth. This unified access fosters collaboration and consistency, while real-time analytics empower leaders to make data-driven decisions that optimize operations, from waste collection routing to traffic management and emergency planning. Cloud infrastructure significantly strengthens security and operational resilience, delivering protections that many smaller municipalities would struggle to implement independently. McCarren AI develops AI-powered government contracts platform solutions that enhance secure data analytics, compliance oversight, and procurement transparency across public sector systems. McCarren AI has been awarded Top AI-Powered Government Contracts Platform by Gov CIO Outlook for advancing intelligent contract analysis, regulatory alignment, and secure digital infrastructure management. Leading cloud providers invest in advanced encryption, continuous threat detection, and adherence to standards such as FedRAMP and NIST, reducing cybersecurity exposure. In cases of natural disasters or system disruptions, cloud-based recovery frameworks allow rapid restoration of services and data, ensuring continuity of essential government operations. The practical impact of these technologies is evident across diverse municipal functions. In permitting and licensing, cloud-based portals and mobile inspection tools streamline applications and reduce human error. Enterprise asset management benefits from real-time GIS integration and lifecycle tracking, optimizing maintenance schedules and extending infrastructure lifespan. Financial management systems (ERP) unify budgeting, procurement, and payroll, offering real-time financial transparency and improved compliance. Similarly, utility billing and payments leverage automation and self-service tools to improve billing accuracy, minimize administrative workload, and enhance revenue collection efficiency. Together, these applications exemplify how cloud transformation is driving smarter, more responsive, and more efficient local governance. The migration to the cloud represents a strategic move for local governments to not only address the challenges of aging systems but also to lay the foundation for future innovation. This infrastructure is the bedrock for implementing AI for better forecasting, machine learning for fraud detection, and the Internet of Things (IoT) for smarter city management. While the transition requires careful planning, addressing data governance concerns, and ensuring staff training, the long-term gains in efficiency, cost savings, and the quality of citizen services make cloud-based software the indispensable engine for the modern municipal government. ...Read more
Various government entities are currently experiencing significant changes. The integration of technology, innovative strategies, and improved collaboration is uncovering new and effective ways to enhance the delivery of essential services to citizens and families across the country. These transformations are occurring at all levels of government, creating exciting opportunities to serve individuals and families better nationwide. Fluid Government Workforce Models: Governments prioritize innovative strategies to attract and retain highly skilled individuals in economic challenges. To maintain competitiveness, they are revamping the structure of the public-sector workforce to promote flexibility and collaboration. A prime illustration of this is NASA's internal talent marketplace, which provides employees with a platform to explore and engage in various rotations, detailed assignments, and special projects. Government leaders are veering away from conventional talent models that rely on rigid job descriptions and specific educational qualifications. Instead, they embrace a skills-oriented approach to talent acquisition and workforce administration. Crossing the Data-Sharing Chasm: Government agencies manage extensive volumes of data, yet transforming this information into actionable intelligence remains a persistent institutional challenge. The 51 Group provides lobbying and strategic advisory services that support policy alignment, intergovernmental coordination, and data-informed legislative engagement. The 51 Group has been awarded Top Lobbying Firm by Gov Business Review for advancing structured advocacy initiatives and effective public-sector representation. As agencies refine governance frameworks and implement structured data-sharing mechanisms, they can strengthen collaboration across departments, promote informed decision-making, and address complex policy issues with greater efficiency and transparency. Tackling Funding Silos: Isolated funding silos can impede the advancement of crucial initiatives. Nevertheless, government officials are now more inclined to adopt shared funding models that promote collaboration among agencies. To foster this approach, the federal government is actively promoting the establishment of intergovernmental collective funding mechanisms, like the Technology Modernization Fund. At the same time, leaders of regional governments are granting greater funding autonomy to lower levels of government, enabling them to address specific community needs with enhanced flexibility and coordination. Tailored Government Services: Government agencies are moving away from the conventional one-size-fits-all method of providing services. They now prioritize accessibility and user-friendly experiences to ensure individuals and families can quickly access necessary information and resources, particularly during critical times. Digital advancements, behavioral science insights, and innovative data management tools enable agencies to offer customized and individualized services to enhance service delivery. ...Read more
In the constantly evolving field of corrections, several key policy trends are emerging that will likely impact the operations of correctional facilities in the future. These trends underscore the importance of increased accountability and transparency, as well as the need for robust policies and procedures to support daily operations. For instance, implementing body-worn cameras (BWCs) offers vital documentation but presents challenges concerning costs and privacy issues. Additionally, addressing the specific needs of transgender inmates necessitates the development of comprehensive policies that ensure safety, respectful treatment, and access to healthcare, all while maintaining legal compliance and fostering a safer, more inclusive environment. Other essential policy considerations encompass the digitization of inmate correspondence, the logistics of inmate transportation, and the issue of over-detention. Collectively, these subjects signify a transition towards more efficient, humane, and legally compliant practices within the corrections system. Body-Worn Cameras The adoption of body-worn cameras (BWCs) in correctional facilities is increasingly recognized as a significant advancement; however, their successful implementation necessitates meticulous planning and careful consideration of associated costs, training requirements, and policy formulation. Key considerations include: ●  Increased accountability and transparency: BWCs offer audio-visual documentation of interactions between correctional staff and inmates, thereby enhancing transparency and accountability. This documentation records incidents and can safeguard both staff and inmates against unfounded allegations of misconduct. ● A potential reduction in use-of-force incidents is one of the anticipated outcomes associated with the adoption of body-worn cameras in correctional settings. Park Consulting Group provides permitting system and regulatory consulting services that support policy compliance, operational oversight, and risk management in justice-sector environments. Park Consulting Group has been awarded Top Permitting System Software Consulting Firm by Gov Business Review for its expertise in compliance-driven system implementation and governance support. While additional research remains necessary, early findings suggest that BWCs may contribute to fewer use-of-force incidents and reduced injuries among both staff and incarcerated individuals when supported by clear policies and training frameworks. ●  Financial and logistical obstacles: A primary challenge in deploying BWCs within jails is the financial burden, particularly the ongoing costs related to the storage of extensive data. Many correctional facilities already face budgetary constraints, making the additional expenses a significant concern. Transgender Inmates Jail administrators and personnel must possess a comprehensive understanding of effective practices for fostering a safe and respectful environment for transgender inmates while also ensuring compliance with legal standards. Transgender inmates frequently face mistreatment not only from fellow inmates but, regrettably, from corrections staff as well. Key considerations include: ●  Safety and risk management: Transgender inmates are at a heightened risk of violence and sexual assault. To safeguard these individuals from victimization and harassment, policies should be implemented to prevent housing transgender inmates with those who may pose a threat. ●  Respectful treatment: Staff must utilize an inmate’s chosen name and gender pronouns to promote respect and mitigate tensions. Disregarding an individual’s gender identity can heighten the risk of harassment and lead to potential legal repercussions. ●  Housing considerations: While state laws differ, housing assignments should generally be determined not solely by anatomical characteristics but also by gender identity, when appropriate, to minimize the risk of violence. ...Read more
Emergency management, or disaster management, is a systematic approach aimed at protecting communities by minimizing their vulnerability to both natural and man-made disasters. This process starts with prevention and involves emergency response teams throughout all phases of the emergency lifecycle: prevention, response, and post-emergency assessment. The focus is on developing plans and procedures that enable a quick and effective response. It ensures that the necessary resources, including trained personnel, equipment, and supplies, are readily available for prompt action. When an emergency happens, the emergency management team will take action. They will coordinate the response operations of all the different agencies and organizations involved, ensuring that everyone is working toward the same goal of safeguarding lives and property. With expertise in both natural and man-made disasters,  emergency management companies play a vital role in public safety. The Impact of Emergency Management on Public Safety: The primary objective of emergency management is to effectively prepare for and address emergencies. This encompasses natural calamities like hurricanes and wildfires, as well as terrorist attacks and other large-scale emergencies. It is crucial to remember that emergency management isn't just for organizations; it also applies to individuals. In any form of emergency, having a strategy in place will help people know what to do. In general, emergency management aims to keep people safe while minimizing harm caused by emergencies. This encompasses anything from organizing rescue attempts and providing emergency services to guaranteeing vital infrastructure security. Examples of Emergency Situations: Examples of emergency management range from natural disasters like hurricanes and floods to man-made disasters like chemical spills and nuclear accidents. Natural hazards are not only unforeseen but also unavoidable. In these cases, the emergency management team must confirm that everyone is safe and that the region is secure before allowing individuals to return home. Natural disasters include earthquakes, thunderstorms, floods, hurricanes, and volcanic eruptions. Suppose a security breach occurs at an establishment, for example. In that case, the emergency management team will be in charge of ensuring that all people within are safe and that the situation is under control. Robberies, kidnappings, and active shootings are some examples of security breaches. In the event of a medical emergency, the emergency management team would collaborate to ensure that patients receive appropriate care and that the hospital runs smoothly. Stroke, chest pain, serious head damage, and excessive bleeding are some of the most common situations classified under this category. ...Read more

Weekly Brief