Carvalho, who unplugged school AI chatbot, wants task force to tell him what went wrong
Los Angeles schools Supt. Alberto Carvalho, who remains determined to bring artificial intelligence into district classrooms despite the collapse of the technology company leading the effort, will appoint a task force to examine what went wrong and how to move forward.
The schools chief announced the task force in an interview with The Times in advance of Tuesday’s annual address to administrators, which is akin to a state-of-the-schools speech.
In his public address, Carvalho is expected to highlight academic progress and L.A. Unified School District initiatives.
In a recent appearance, he said he was hopeful that standardized test scores would rise at all grade levels in math and English. Although school districts throughout the state have received results — and can make them public if they wish — the state has not yet released local or statewide scores.
The star attraction at last year’s speech, “Ed” the chatbot, is not expected to be spotlighted. Carvalho had touted Ed as an artificial intelligence-enhanced student advisor that was to be a component of a unique Individual Acceleration Plan, or IAP, for every student.
The full rollout of the IAP is on hold. And it’s hard to find students, teachers or other staff who have used any part of the system since its official launch. It’s not available at most schools.
Moreover, the much-publicized chatbot has been unplugged entirely in the wake of the apparent financial collapse of Boston-based AllHere Education, the company that provided the technology.
LAUSD unplugged chatbot after collapse of company that created it, but Carvalho says other features remain to help students. Most schools don’t yet have it.
Carvalho said he would assemble independent experts to conduct a wide-ranging review, with the goal of renewing the effort. One key focus will be whether the district could strengthen its bidding process and oversight to prevent a similar scenario in the future.
“I think it is time to slow the process down, really get a better understanding, a deep understanding through the eyes of independent experts, before we move forward at 100% utilization of the tool,” Carvalho said in the interview. All the same, “this was a bold vision that resulted in a product that works. It’s been attested to by everybody I’ve spoken with.”
After the preview of “Ed” in the speech a year ago, the district presented the finished chatbot in March. The computer tool was symbolized by an animated sun named Ed. Also part of the package was an online platform — it, too, was called Ed — that was to combine all information about students and district resources into one place. The system was meant to be interactive and even assertive — able to make academic recommendations to students, and provide helpful nudges if a student is late for a bus or absent from class too often.
Less than three months later, AllHere laid off the “vast majority” of its employees, as one senior manager put it. The company has not responded to inquiries and its future is in doubt. District officials say they have no understanding of what befell the company.
“We all woke up to a reality where the world changed overnight without us knowing why,” Carvalho said of the situation with AllHere. “And we still really don’t know why. And that’s exactly what I’m seeking.”
L.A. Unified has paid AllHere $3 million of a potential $6-million contract, and district officials said all the payments to date were appropriate for work performed. But no future payments will be made under the contract.
Carvalho anticipates that the task force, once assembled, will complete its work over about three months.
The task force will have much to dig into, including allegations from former AllHere manager Chris Whiteley, who left the company in April, not long before widespread layoffs.
LAUSD sidelines “Ed,” an AI chatbot, after a splashy kick-off featuring a company that has now tanked. District also is dealing with another data breach.
In an interview, Whiteley, who had been the company’s senior director of engineering, noted areas of concern.
Whiteley said the company had taken shortcuts with security and data privacy in violation of sound practice, school district policy and its contract with L.A. Unified. One such violation was using overseas servers to process data, he said.
However, the former AllHere employee said he knew of no data breach that had exposed the personal information of students, their families or staff. The potential danger was real, he said, but only theoretical, as far as he knew.
Whiteley also contends that the chatbot has an important design flaw, which he called the “overuse” of information. When a student signs in with a password and asks a question, for example, the chatbot pulls in all available personal and private information in the L.A. Unified system about the student and that student’s family.
This happens even in response to a simple “Hello.”
Any overuse of information represents an unnecessary, enhanced security risk for the release of private data, according to Whiteley and experts interviewed by The Times.
Such flaws can be fixed, Whiteley said, but AI technology also has advanced in the last year and it might be more efficient to start over rather than to fix things.
Whiteley said he explained his concerns in an early July interview with investigators from the school district’s inspector general’s office, as first reported in the 74, an education news site.
“I’ve read the allegations,” Carvalho said. “I’m disturbed by the allegations.” If true, the company’s conduct “would be contrary to what was reflected in our contract. There is specific language in the contract that would not allow for some of those shortcuts being made. That is one of the areas that the task force will be looking into.”
AllHere has not responded to efforts to reach the company, which included questions to a senior manager and its communications office.
But a company representative has responded to L.A. Unified, Carvalho said: “We’ve had — our team has had — conversations with the company about those allegations. The company has denied those allegations.”
Sign up for Essential California
The most important California stories and recommendations in your inbox every morning.
You may occasionally receive promotional content from the Los Angeles Times.
