The Black Box: Colleges spend thousands on AI to prevent suicides and shootings. Evidence that it works is scant.
hen Social Sentinel representatives pitched their service to Florida’s Gulf Coast State College in 2018, they billed it as an innovative way to find threats of suicides and shootings posted online.One tweet notified the school about a nearby fishing tournament: “Check out the picture of some of the prizes you can win - like the spear fishing gun.”: “Can we pretend that airplanes in the night sky are like shooting stars? I could really use a wish right now.
As schools and universities confront a worsening mental health crisis and an epidemic of mass shootings, Social Sentinel offers an attractive and low-cost way to keep students safe. But experts say the service also raises questions about whether the potential benefits are worth theRecords show Social Sentinel has been used by at least 38 colleges in the past seven years, including four in North Texas.
Privacy and legal experts say the service may give schools the false impression that technology can help avert tragedies, while potentially exposing them to even greater liability. The employee, who asked not to be named for fear of retaliation, said problems with the service were an open secret at the company, and described it as “snake oil” and “smoke and mirrors.”also contacted more than two dozen other former company employees, who either did not respond or said they had signed nondisclosure agreements preventing them from speaking publicly about their time at the company.
Ed Reynolds, police chief at the University of North Texas, defended the system, but also estimated that “99.9 percent were messages we didn't need to do anything with.” After using the service for about three years, UNT ended its contract with the company in November 2018. Guilbault also said the company has seen multiple instances of the service working to prevent students from harming themselves or others, but declined to discuss specific clients.
Kennesaw State initially used the service often. But documents show the university gradually stopped checking its alerts. From April to September 2020, it viewed only 34% of the alerts it was sent, records show. The former Social Sentinel employee said it was so common for schools to not check their alerts that company workers deleted particularly irrelevant ones every day without schools ever noticing.
The former Florida State chief appeared to share the same story in a testimonial in Social Sentinel marketing material. The word “trump” showed up at least 75 times, making it one of the 100 most common words in the posts, after removing common terms like “the” “it” “and” or “but.”“How long, O Lord, wilt thou look on?” one tweet sent to Palm Beach State college read. “Rescue me from their ravages, my life from the lions!”
Studies show even the most sophisticated AI language models can perpetuate racial and gender biases if they were trained on text that contained such prejudices. Less than two years later he sat behind bars in a Phoenix jail cell, charged with a making a terrorist threat.In 2016, Robinson accepted an offer from Arizona State, forgoing 26 other full scholarships from elite private schools and football powerhouses alike. The school’s coaches praised both his football prowess and high grades, which landed him in the university’s honors college.
Shortly after sending the tweets, Robinson said a friend told him the police were outside of the apartment where he was staying. He peered out the window and saw men dressed in black retrieving AR-15 rifles from their cars. As more than 60 officers from ASU and Tempe’s police departments converged on the building, Robinson called his uncle on FaceTime. He expected it to be the last friendly voice he ever heard.
“Any time you try to turn this into a small data problem, it’s a problem,” said Coppersmith, who also co-authored several papers on using machine learning models for suicide prevention. “A single message, a single user, a single type, a single whatever is more prone to error, especially if you're trying to look at the things that are highest risk.”
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
As enrollment falls and skepticism grows, some colleges are cutting pricesThe pace of annual increases in tuition and fees has for the first time since the early 1980s slowed to a rate that’s well below inflation. Now some higher education institutions are starting to lower their prices.
Read more »
Community colleges scramble to find students amid enrollment crisis - Jacksonville Business JournalWith 827,000 fewer students since the start of the pandemic, the nation’s community colleges are scrambling to find students.
Read more »
Prosecutor: Evidence shows Trump ‘explicitly’ OK’d tax fraudThe Trump Organization is accused of helping executives avoid paying income taxes on company-paid perks.
Read more »
Early humans domesticated themselves, new genetic evidence suggestsWhen humans began to tame dogs and sheep and cattle, they may have continued a tradition that started with a completely different animal: us. ScienceMagArchives
Read more »
Idaho murders: Handprint spotted next to evidence tape at house where four students were killedAt the Moscow, Idaho home where four University of Idaho students were killed, a hand print can be seen next to an evidence tape on the window of the King Road house.
Read more »
Prosecutor: Evidence shows Trump 'explicitly' sanctioned alleged tax fraud scheme at two companiesFormer President Donald Trump 'explicitly' sanctioned the alleged tax fraud scheme at the heart of the criminal trial of two Trump companies, a prosecutor argued Friday, citing trial evidence.
Read more »