Publications

Showing 11 results for Author: Amir Sadovnik

View All publications

  • Jul, 2026

    Conference Paper

    FATHOMS-RAG: A Framework for the Assessment of Thinking and Observation in Multimodal Systems that use Retrieval Augmented Generation

    Retrieval-augmented generation (RAG) has emerged as a promising paradigm for improving factual accuracy in large language models (LLMs). We introduce a benchmark designed to evaluate RAG pipelines as a whole, evaluating a pipelines ability to ingest several modalities of information. We present (1) a curated dataset of 93 questions designed to evaluate a pipeline's ability…

  • Jul, 2026

    Conference Paper

    3D reconstruction and neural rendering for adversarial machine learning

    While evasion attacks on computer vision systems have been widely studied, creating attacks that remain effective under significant changes in viewpoint continues to be challenging. Traditional approaches often rely on affine transformations of images, but these approaches degrade at larger perspective shifts and often produce unrealistic or ineffective perturbations. Rece…

  • Mar, 2026

    Conference Paper

    Semantic Stealth: Crafting Covert Adversarial Patches for Sentiment Classifiers Using Large Language Models

    Deep learning models have been shown to be vulnerable to adversarial attacks, in which perturbations to their inputs cause the model to produce incorrect predictions. As opposed to adversarial attacks in computer vision, where small changes introduced to pixel values can drastically alter a model's output while remaining imperceptible to humans, text-based attacks are diff…

  • Mar, 2026

    Conference Paper

    On the Abuse and Detection of Polyglot Files

    A polyglot is a file that is valid in two or more formats. Polyglot files pose a problem for file-upload and generative AI web interfaces that rely on format identification to determine how to securely handle incoming files. In this work we found that existing file-format and embedded-file detection tools, even those developed specifically for polyglot files, fail to relia…

  • Mar, 2026

    Conference Paper

    Towards a High Fidelity Training Environment for Autonomous Cyber Defense Agents

    Cyber defenders are overwhelmed by the frequency and scale of attacks against their networks. This problem will only be exacerbated as attackers leverage AI to automate their workflows. Autonomous cyber defense capabilities could aid defenders by automating operations and adapting dynamically to novel threats. However, existing training environments fall short in areas suc…

  • Mar, 2026

    Conference Paper

    Hiding-in-Plain-Sight (HiPS) Attack on CLIP for Targetted Object Removal from Images

    Machine learning models are known to be vulnerable to adversarial attacks, but prior works have mostly focused on single-modalities. With the rise of large multi-modal models (LMMs) like CLIP, which combine vision and language capabilities, new vulnerabilities have emerged. However, these multimodal targeted attacks aim to completely change the model's output to what the a…

  • Mar, 2026

    Conference Paper

    Assessing Membership Inference Attacks under Distribution Shifts

    Membership inference attacks (MIAs) exploit machine learning models to infer whether a data point was in the training set, posing significant privacy risks even with limited black-box access. These attacks rely on the attacker approximating the target model’s training distribution, yet the impact of distribution shifts between target and shadow models on MIA success remain…

  • Mar, 2026

    Conference Paper

    The Effects of Compounded Model Size Reductions on Adversarial Robustness

    Recent advances in Edge AI and Tiny Machine Learning (TinyML) have enabled the deployment of machine learning models on resource-constrained environments. However, deploying these models on edge devices, such as micro-controllers, requires significant model footprint reduction through a variety of techniques such as quantization, pruning, and clustering. While these optimi…

  • Feb, 2026

    Book Chapter

    Bridging the Gap from Research to Reality: Methods for Fortifying Mitigation Measures Against Adversarial AI

    While research in the area of Adversarial AI and Mitigation (AAI&M) has quickly grown, research toward investigating the physical realizability of these methods has lagged behind. AAI&M Research often focuses on constrained or 2-Dimensional scenarios that don’t have direct translation to “in-the-wild” real-world problems. As example, an adversarial patch is often only gene…

  • Oct, 2025

    Journal

    The Path to Autonomous Cyberdefense

    Defenders are overwhelmed by attacks against their networks, which will only be exacerbated as attackers leverage artificial intelligence to automate workflows. We propose a path to autonomous cyberagents able to augment defenders by automating critical steps in the cyberdefense lifecycle.

  • Mar, 2025

    Journal

    Lantern: An Interactive Adversarial AI Workbench

    Lantern, an innovative adversarial artificial intelligence (AI) workbench with unique features, is presented in this article. Lantern allows users to evaluate adversarial attacks quickly and interactively against artificial intelligence (AI) systems to protect them better. While there has been extensive work in assessing the robustness of AI models against attacks at a lar…