Tracked: How Colleges Use AI To Monitor Student Protests

The pitch was enticing and easy.

For a couple of thousand {dollars} a yr, Social Sentinel provided colleges throughout the nation refined know-how to scan social media posts from college students susceptible to harming themselves or others. Used appropriately, the software may assist save lives, the corporate stated.

For some faculties that purchased the service, it additionally served a unique objective — permitting campus police to surveil scholar protests.

Throughout demonstrations over a Accomplice statue at UNC-Chapel Hill, a Social Sentinel worker entered key phrases into the corporate’s monitoring software to seek out posts associated to the protests. At Kennesaw State College in Georgia 5 years in the past, authorities used the service to trace protesters at a city corridor with a U.S. senator, information present. And at North Carolina A&T, a campus official advised a Social Sentinel worker to enter key phrases to seek out posts associated to a cheerleader’s allegation that the college mishandled her rape criticism.

An investigation by The Dallas Morning Information and the Investigative Reporting Program at UC Berkeley’s Graduate Faculty of Journalism reveals for the primary time that as extra college students have embraced social media as a digital city sq. to precise opinions and arrange demonstrations, many school police departments have been utilizing taxpayer {dollars} to pay for Social Sentinel’s companies to watch what they are saying. At the least 37 faculties, together with 4 in North Texas, collectively educating lots of of 1000’s of scholars, have used Social Sentinel since 2015.

As a nonprofit journalism group, we rely in your assist to fund greater than 170 reporting tasks yearly on important international and native points. Donate any quantity at the moment to change into a Pulitzer Heart Champion and obtain unique advantages!

The true variety of faculties that used the software might be far greater. In an e mail to a UT Dallas police lieutenant, the corporate’s co-founder, Gary Margolis, stated it was utilized by “lots of of schools and universities in 36 states.” Margolis declined to touch upon this story.

The Information examined 1000’s of pages of emails, contracts and advertising and marketing materials from faculties across the nation, and spoke to highschool officers, campus police, activists and specialists. The investigation exhibits that, regardless of publicly saying its service was not a surveillance software, Social Sentinel representatives promoted the software to universities for “mitigating” and “forestalling” protests. The paperwork additionally present the corporate has been transferring in a brand new and doubtlessly extra invasive course — permitting colleges to watch scholar emails on college accounts.

For faculties struggling to answer high-profile faculty shootings and a worsening campus psychological well being disaster, Social Sentinel’s low-cost software can appear to be a great deal. Along with the handfuls of schools that use the service, a Information investigation final yr revealed that a minimum of 52 faculty districts in Texas have adopted Social Sentinel as an extra safety measure since 2015, together with Uvalde CISD the place a gunman killed 19 kids and two lecturers in Might. The corporate’s present CEO J.P. Guilbault additionally stated their companies are utilized by one in 4 Okay-12 colleges within the nation.

Some specialists stated AI instruments like Social Sentinel are untested, and even when they’re adopted for a worthwhile objective, they’ve the potential to be abused.

For public faculties, using the service units up an extra battle between defending college students’ Constitutional rights of free speech and privateness and colleges’ obligation to maintain them secure on campus, stated Andrew Guthrie Ferguson, a legislation professor at American College’s Washington Faculty of Regulation.

“What the know-how means that you can do is establish people who’re related collectively or are related to a spot or location,” stated Ferguson. “That’s clearly considerably chilling for First Modification freedoms of people that consider in a proper to protest and dissent.”

Navigate360, the personal Ohio-based firm that acquired Social Sentinel in 2020, known as The Information’ investigation “inaccurate, speculative or by opinion in lots of situations and considerably outdated.” The corporate additionally modified the identify of the service from Social Sentinel to Navigate360 Detect earlier this yr.

Picture by Ari Sen/The Dallas Morning Information.

In an interview in February, Guilbault stated the service doesn’t have the power to watch protests.

However paperwork from Kennesaw State present campus police tracked demonstrators’ on-line exercise for days with Social Sentinel earlier than a contentious 2017 city corridor.

Brandy White, a prison intelligence analyst in KSU’s police division, was in control of the monitoring. On instruction from her supervisors, White entered details about demonstrators and protest teams into Social Sentinel’s monitoring software and arrange searches to seek out posts in regards to the occasion, emails present.

White additionally acquired a KSU police intelligence briefing from a colleague in regards to the occasion. The doc, obtained by The Information in response to a public information request, singled out one progressive activist group, the liberal grassroots community Indivisible, and cited conservative conspiracy theories that George Soros funded the protesters.

The report additionally recognized a younger progressive activist named Marisa Pyle.

On the time, Pyle was a 19-year-old scholar at a close-by school. She advised The Information that she had no thought about KSU’s monitoring.

“There is not any model of that the place I wanted any type of particular consideration or surveillance by Kennesaw State.”

“An ear to the social panorama”

The concept for Social Sentinel got here out of a police briefing within the mid-2000s.

A sergeant named Jimmy had extra to say than traditional.

He advised Margolis, then the police chief on the College of Vermont in Burlington, a few sequence of petty thefts, a sexual assault and a protest deliberate on the govt workplaces of the college.

“Jimmy, the place did we get this info?” he requested.

The sergeant despatched Margolis to see an in a single day dispatcher, who confirmed him the supply — new on-line websites, together with an internet site known as The Fb.

It was in that second, Margolis later stated, the seed for Social Sentinel was shaped.

“It grew to become clear that we would have liked a digital beat cop – ‘somebody’ with an ear to the social panorama,” he wrote in a 2015 weblog submit on the Social Sentinel web site.

In 2008, in his last yr as police chief, Margolis merged his consulting service with one other began by Steven Healy, a former navy police officer who beforehand held high security roles at Princeton and Wellesley. 4 years later, the 2 introduced Campus Sentinel, an app that aggregated crime stats on school campuses.

It could take one other two years earlier than Margolis may understand his imaginative and prescient for an app that stored tabs on on-line exercise. In 2014, with assist from a Boston-based personal fairness agency and a New York-based enterprise capital firm, the duo rebranded the service as Social Sentinel, a social media risk alert service.

The College of North Texas was an early adopter.

The college’s police chief, Ed Reynolds, advised The Information final yr that the college scheduled a presentation with the corporate after listening to success tales from different chiefs at a convention.

The presenters billed Social Sentinel as an AI software that would detect threatening tweets in regards to the UNT campus, and confirmed Reynolds the way it might be used to seek out posts.

After contemplating different competing companies, the UNT police chief signed a $15,000-a-year contract in December 2015.

In November of that yr, a Social Sentinel worker despatched Reynolds solutions for methods to use the service, emails obtained by The Information present. In a single part, the corporate proposed the college undertake a rule stopping them from monitoring protests, besides once they threatened public security.

It’s unclear whether or not UNT adopted the recommended coverage — no signed copy exists within the information offered to The Information. Apart from the proposed procedures despatched to the North Texas faculty, Social Sentinel appears to have provided faculties few guardrails for methods to use its service. The Information reached out to each college recognized to make use of the service — not one of the faculties that responded stated they’d guidelines particularly governing Social Sentinel.

Reynolds stated UNT stopped utilizing the service in 2018, after figuring out they might deal with the monitoring in home. That very same yr, Collin Faculty signed a contract with Social Sentinel and UT Dallas renewed its contract for an additional three years.

At the least one school in North Texas nonetheless makes use of the monitoring software. Nicole Shaw, the police chief at North Central Texas Faculty, stated the college’s newest contract for the service value $12,750 a yr.

Shaw stated she knew of just one occasion the place the service alerted to a doubtlessly troubling scenario. In that case, the division was already conscious of the scholar who wrote the submit and the scholar was not an imminent risk to himself.

Doubtlessly suicidal college students aren’t the one factor the school is monitoring for.

“Protests, that’s one of many buzzwords,” she stated.

“We do not take sides, we simply need to be certain that everybody has a secure expertise.”

The whitepaper

Social Sentinel has lengthy stated its service can’t be used for monitoring protests.

However emails in 2017 between campus police officers at Kennesaw State present the college used the service a minimum of 3 times for that objective.

Along with the city corridor assembly, the college monitored protests towards President Trump’s 2017 ban on vacationers from a number of Muslim-majority international locations and demonstrations towards the college’s former chancellor, Sam Olens.

A number of spokespeople from Kennesaw State declined repeated requests for interviews for this story.

Paperwork present the service was a key software for the college’s police division in 2017. Members of the employees reported what number of tweets have been flagged of their month-to-month conferences that yr. The service confirmed up in a 2017 job description outlining the every day duties of a prison intelligence analyst, saying the function included monitoring Social Sentinel for “crime, civil unrest, protests” and extra.

Guilbault stated the service doesn’t have the power to watch protests, however declined to touch upon any actions earlier than their acquisition of the corporate in 2020.

“The phrase protest shouldn’t be even in our engines,” he stated, referring to the know-how the corporate says it makes use of to investigate social media posts.

Paperwork The Information obtained from colleges across the nation present firm representatives repeatedly promoted Social Sentinel’s capacity to watch protests.

In 2017, a marketer working with the corporate, David Smitherman, reached out to a UNC-Charlotte administrator, Christine Davis, to tout the service.

Learn the entire doc right here. Picture courtesy of The Dallas Morning Information.

Smitherman had contacted Davis a minimum of two different instances saying the service might be used for forestalling doubtlessly risky protests, emails present. In a November 2016 message to a different administrator, he wrote that it might be used for mitigating the impacts of “disruptive demonstrations & protests.” Across the identical time, representatives used the identical language of “mitigating” and “forestalling” protests in messages to Stephen F. Austin State College and UNC-Asheville.

This time he added one thing totally different — a whitepaper from the corporate titled “Demonstrations and Protests: Utilizing Social Media to Collect Intelligence and Reply to Campus Crowds.”

The doc outlines how the service can be utilized to watch earlier than and through demonstrations, and in addition mentions that Michigan State College used the service to watch protests. Emails from 2015 and 2016 obtained by The Information additionally present the whitepaper was despatched to a minimum of three different faculties — UT Dallas, Appalachian State and UNC-Asheville — contemplating the service.

“Throughout an occasion, risk alerts can present vital perception in regards to the leaders or agitators who could need a confrontation with legislation enforcement, the final local weather of the gang and the potential for crowd progress,” the doc says.

Davis advised The Information that she didn’t observe up with Smitherman and the college didn’t search a service to watch protests. Smitherman declined to remark for this story.

“Monitoring is Icky”

From the start, Social Sentinel’s co-founder, Margolis, stated publicly that the service couldn’t be used to surveil particular person individuals.

In emails to purchasers, the corporate had a unique message.

“I hear that you’re taken with importing usernames, consumer accounts, and so on to observe recognized threats,” one worker wrote to former North Carolina State College Police Chief Jack Moorman in August 2015. “We just lately launched that characteristic, and now you can add a listing of Twitter authors.”

“I recognize the data on the characteristic of importing the accounts of people that we need to observe,” Moorman wrote again. “This ought to be an important profit to ongoing risk evaluation instances.”

NC State declined to touch upon Moorman’s actions, however stated Social Sentinel was not used to focus on people or teams. Moorman didn’t reply to requests for remark.

Paperwork present colleges could have used one other characteristic of the service known as the Native+ Record to focus on protesters and particular person college students. Of the 37 faculties recognized by The Information as utilizing the service, solely two, Oregon State and the College of Connecticut, offered their Native+ Record in response to public information requests.

At the least one North Texas school — UT Dallas — may have used the characteristic to watch people. Vanessa Balderrama, a undertaking coordinator for the college, stated the college couldn’t launch its Native+ Record as a result of it contained college students’ names.

North Carolina A&T, a traditionally Black school in Greensboro, could have used the checklist characteristic in a means that may reveal posts important of the college’s management after a cheerleader stated her coaches mishandled her rape criticism. Emails from the college in February 2019 present a Social Sentinel worker suggesting a lengthy checklist of key phrases and phrases associated to the protests to the college’s social media supervisor.

“Sure, please add these in,” the supervisor wrote in response.

North Carolina A&T Affiliate Vice Chancellor Todd Simmons denied utilizing the service to watch the demonstrations, saying the e-mail was “an effort to upsell us to the Native Plus product.” In its investigation, The Information discovered no proof that the Native+ Record value colleges extra money.

The Native+ Record resembles one other software standard with legislation enforcement across the identical time — the CIA-funded service Geofeedia.

The North Carolina faculties have been accustomed to the competing software — a minimum of two of the faculties’ police departments have been a part of a legislation enforcement group run by North Carolina’s State Bureau of Investigation, which had a contract with Geofeedia.

Paperwork obtained by The Information present UNC-Chapel Hill’s police division requested the state bureau to surveil professional and anti-abortion protesters in October 2015. The investigative company rapidly obliged. Emails between the companies present SBI tracked key phrases and hashtags together with “Feminist College students United,” “College students for Reproductive Justice,” “#feminist,” “#studentsforlife” and even “#unc.”

In 2016, the ACLU of Northern California disclosed that legislation enforcement companies had used Geofeedia to watch Black Lives Matter demonstrators in Ferguson. Quickly after, Twitter, Fb and Instagram introduced they might droop or restrict Geofeedia’s entry to their service. A month later, Geofeedia laid off greater than half of its employees.

The ACLU revelations raised alarms at Social Sentinel. One former worker advised The Information that executives determined to cease promoting the service to native police departments after they realized their service was additionally used to watch Black Lives Matter protesters.

In a 2017 webinar, Margolis referenced the issues with Geofeedia and stated the corporate shared the ACLU’s fears.

“Our group at Social Sentinel had precisely those self same considerations again in 2015, and we put a stake within the floor and created the risk alert strategy for precisely the explanations the ACLU cited,” Margolis advised the webinar’s attendees. “Focusing on and surveilling people or teams shouldn’t be OK.”

“Monitoring is icky,” Margolis added.

Demonstrators at UNC-Chapel Hill protest the Silent Sam statue in August 2018. Photos by Ari Sen. United States, 2018.

Demonstrators at UNC-Chapel Hill protest the Silent Sam statue in August 2018. Photos by Ari Sen. United States, 2018.

After the disclosures about Geofeedia, UNC-Chapel Hill sought one other social media monitoring service, and finally signed a $24,500 yearly contract with Social Sentinel in November 2016.

Lower than a yr later, the college surveilled scholar protesters once more.

NBC Information reported that the school collaborated with an FBI Joint Terrorism Activity Drive to surveil the social media posts of individuals protesting a Accomplice statue on campus in 2017. It’s unclear which monitoring service the college utilized in that case.

Throughout one other protest of the Accomplice monument a yr later, campus police once more monitored posts, this time utilizing Social Sentinel’s checklist characteristic, emails present.

The Information acquired no documentation from the school exhibiting UNC used the service for something however monitoring protests. The previous campus police chief, David Perry, and Chancellor Kevin Guskiewicz stated in 2020 that they weren’t conscious of any situations when the service had helped to forestall violence on the campus.

“The entire haystack”

Some college officers stated they weren’t bothered by the potential privateness implications of Social Sentinel as a result of they stated they solely used the software to watch public social media posts.

“I do not suppose it was ever a priority of mine,” Reynolds, the UNT Chief, advised The Information final yr. “The data that we have been was public info.”

“My understanding of the product is they’ll solely draw these feedback from public posts on social media,” stated David Thomasee, govt director of operations at Gulf Coast State Faculty. “I am undecided that the expectation of privateness is there if anybody on the earth can see it.”

However even gathering publicly posted info could hurt college students’ privateness and speech rights, specialists say. Jake Wiener, a lawyer on the Digital Privateness Info Heart additionally stated social media monitoring companies could lengthen the attain of legislation enforcement companies past what they have been beforehand in a position to observe.

“One thing like a social media surveillance program goes to comb in a lot information,” Wiener stated. “An officer goes to be much more prone to examine, and probably arrest, people who find themselves not truly the varieties of targets they’re searching for.”

Ferguson, the legislation professor at American who additionally authored The Rise of Large Information Policing, stated using social media monitoring instruments might be challenged beneath an interpretation of a 2017 Supreme Court docket Case, Carpenter v. U.S. In that case, the court docket dominated that the federal government couldn’t seize a suspected robber’s cellular phone information, together with location information from cell towers, with no warrant.

The court docket’s choice could mirror a shift within the interpretation of privateness legislation to a brand new give attention to how a lot info was collected quite than simply what info was collected.

“You are searching for a needle within the haystack however you have to gather the entire haystack to seek out the needle,” Ferguson stated.

Universities may additionally have extra incentives to undertake surveillance know-how than native police.

“We undoubtedly hear directors speaking so much like ‘the college is sort of a household and we’re right here to guard you from something dangerous that may occur on campus,’’’ stated Alex Morey, a lawyer on the Basis for Particular person Rights and Expression.

“That type of mentality would result in a heightened willingness to see these surveillance methods as a great factor.”

Electronic mail Monitoring

4 months after a gunman killed two individuals and injured 4 others at UNC-Charlotte in 2019, Judy Stanwyck seen one thing unusual.

Stanwyck, a senior purchaser for the college, was dealing with UNC-Charlotte’s buy of Social Sentinel. She wasn’t accustomed to the know-how so she regarded up the corporate’s web site and began studying.

One characteristic jumped out — the corporate was promoting the power to watch college students’ college e mail accounts.

“I discovered this,” Stanwyck wrote in an e mail to her colleague, together with the hyperlink to the web site. “I’d simply affirm precisely what they’re buying.”

Stanwyck didn’t reply to requests for remark.

Within the firm’s proposed contract with UNC-Charlotte, it stated the e-mail monitoring service was solely accessible for Okay-12 colleges. However in an e mail to the college’s affiliate vice chancellor for security and safety, an worker stated they might add that service in the event that they wished to.

The corporate initially stated it solely monitored public info on social media. However since a minimum of 2019, information present it has provided faculties the power to watch personal scholar emails.

It’s unclear whether or not a university has taken the corporate up on its supply — The Information was unable to discover a college that had used the e-mail monitoring service, although the follow is frequent in Okay-12 colleges.

Privateness specialists stated they have been appalled by the service’s capacity to watch personal scholar messages.

“The concept of including surveillance software program to the campus infrastructure the place college students are alleged to be taught is simply stunning,” Dave Maass, the director of investigations on the Digital Frontier Basis, stated. “It is simply an abomination to the thought of scholastic freedom.”

UNC-Charlotte in the end didn’t signal the contract, citing price range limitations. On the identical time, the corporate was selling the e-mail monitoring service to a different North Carolina school — UNC-Asheville.

In 2019 emails to the college, the co-founder Margolis promoted Social Sentinel’s capacity to work with the Google suite of companies and promised they might work with Microsoft Outlook within the close to future.

One other firm worker marketed the brand new Gmail integration within the signature of each e mail she despatched to the college that yr.

The Navigate360 CEO Guilbault stated some faculties use the service to watch emails. However they will not be stopping there.

In emails to Gulf Coast and Palm Seaside state faculties in 2019, the corporate touted its capacity to watch Google Docs, Google Hangouts chats and even Fb Messages.