Concerns over AI surveillance in schools are intensifying after armed officers swarmed a 16-year-old student outside Kenwood High School in Baltimore when an AI gun detection system falsely flagged a Doritos bag as a firearm.

Allen was handcuffed at gunpoint. Police later showed him the AI-captured image that triggered the alert. The crumpled Doritos bag in his pocket had been mistaken for a gun.

  • Colloidal@programming.dev
    link
    fedilink
    arrow-up
    35
    ·
    1 day ago

    the system “functioned as intended,” […] Baltimore County Public Schools echoed the company’s statement

    “We want a system that sends armed cops primed to kill a black student”, says the community elected school board. “We believe that’s what our racist constituents want”, they continued.

  • CerebralHawks@lemmy.dbzer0.com
    link
    fedilink
    English
    arrow-up
    16
    arrow-down
    1
    ·
    1 day ago

    It sure looked like a gun in the message, assuming that was the image that was sent out and not a dramatisation.

    Also, I think it worked exactly how it was supposed to. It SWATted a person of colour. Par for the course. As it was trained to do.

    • calliope@retrolemmy.com
      link
      fedilink
      arrow-up
      4
      ·
      1 day ago

      That is a marketing image from the gun-identifying software company and isn’t actually related to the article.

      The article said he was outside and the crumpled bag of Doritos was hanging out of his pocket. He wasn’t brandishing it like a firearm.

    • brvslvrnst@lemmy.ml
      link
      fedilink
      arrow-up
      5
      ·
      edit-2
      1 day ago

      I think that’s a dramatization, based on the subtitle there

      Edit:

      This looks fake as fuck lol

  • Fluffy Kitty Cat@slrpnk.net
    link
    fedilink
    English
    arrow-up
    10
    ·
    edit-2
    1 day ago

    It’s unjust to force children and Teens to go to schools that use these creepy tools. They must be removed before a student gets killed by cops because of it

  • hodgepodgin@lemmy.zip
    link
    fedilink
    arrow-up
    1
    ·
    edit-2
    17 hours ago

    From the image, it didn’t identify the Doritos as a gun. It was the wall’s bottom black paint at a corner that looked like a gun. This is such an interesting case of where malice and stupidity cross, because imagine being afraid of your students this much and being too stupid to know that a system like this would have false positives often. Imagine a tableau of probabilities:

    ———————Is Gun | Not gun

    Thinks gun | 0.95 | 0.005

    Thinks not gun | 0.05 | 0.995

    Seems good right?

    Well, let’s say this system takes 100,000 pictures in a year. Let’s say we’re in a bad neighborhood and 10 separate pictures had a gun this year. So 99,990 pictures didn’t. Going by this table, only 9.5 out of 508.95 students will be correctly identified as threats. That’s is a false positive rate of 98%. To stupid people in power, this is the system “working as designed.” The false negative rate is decent. It ignores a gun in 0.5 pictures and ignores students in 99,490 pictures, so very close to 0.

    Lmk if my math seems off

  • Remember_the_tooth@lemmy.world
    link
    fedilink
    arrow-up
    8
    ·
    1 day ago

    This is awful, but if we’re going to do futuristic dystopia, we should have more fun with it. Along those lines, it should have been a bag of Cheetos for being “dangerously cheesy.”

    This infringement on civil rights brought to you by Frito-Lay, a subsidiary of Pepsi Co., and chewers like you.

    • Cris@lemmy.world
      link
      fedilink
      English
      arrow-up
      5
      ·
      edit-2
      1 day ago

      You know that vintage meme you’d see hanging on pub walls: “drink coffee! Do stupid things faster and with more energy!”?

      I guess ai surveillance is like that but for police ineptitude. Goody!