Arista arranged for Smith to begin recording her debut album in August 1975. At Smith's suggestion, the label planned to book studio time for Smith with producer Tom Dowd at Criteria in Miami, but Dowd's close association with rival label Atlantic Records stalled these plans. Smith had a change of heart and instead set out to enlist Welsh musician John Cale, formerly of the New York City rock band the Velvet Underground, to produce Horses after she was impressed by the raw sound of his solo albums, such as 1974's Fear. Cale accepted, already familiar with Smith's band and being acquainted with the band's bassist Ivan Král.
Horses was recorded at Electric Lady Studios in New York City, with Smith retaining the same backing band with whom she performed live at the time—Jay Dee Daugherty on drums, Lenny Kaye on guitar, Ivan Král on bass guitar, and Richard Sohl on keyboards. For the album, Smith and her band recorded several songs that were already fixtures of their sets at CBGB, including "Gloria", "Redondo Beach", "Birdland", and "Land". Allen Lanier of Blue Öyster Cult and Tom Verlaine of Television participated in the recording sessions as guest musicians, performing on the songs "Elegie" and "Break It Up", respectively.
The first studio session was held on September 2. Cale later recalled that the band initially "sounded awful" and played out of tune due to their use of damaged instruments, compelling him to procure the band new instruments before commencing recording. The differences between the work process of Cale, who was an experienced recording artist, and the process of Smith, who at that point was primarily a live performer, became apparent in the early stages of recording and were a source of tension between the two artists, who frequently clashed in the studio. Lenny Kaye also highlighted the irreconcilable differences between their musical visions for the album, with Cale picturing "a more arranged record, one fleshed out with intriguing sound palettes and melodic lines", and Smith and her band preferring a more spontaneous approach to playing their material, akin to their live performances. The final album was ultimately informed by both perspectives, making use of multitracking and overdubbing on its more structured songs, while still capturing the musical improvisation that typified the band's live act. Cale wished to augment the band's approach on certain songs with string instruments, but Smith vehemently opposed this idea. Lanier, who was Smith's boyfriend at the time, did not get along with Cale, nor—particularly so—with Verlaine, who had previously dated Smith. This tension culminated with Lanier and Verlaine getting into a physical altercation during the final session, held on September 18.
For several years after the album's release, Smith often downplayed Cale's contributions to Horses and suggested that she and her band had ignored his suggestions entirely. In a 1976 interview with Rolling Stone, Smith described her experience:
My picking John was about as arbitrary as picking Rimbaud. I saw the cover of Illuminations with Rimbaud's face, y'know, he looked so cool, just like Bob Dylan. So Rimbaud became my favorite poet. I looked at the cover of Fear and I said, 'Now there's a set of cheekbones.' In my mind I picked him because his records sounded good. But I hired the wrong guy. All I was really looking for was a technical person. Instead, I got a total maniac artist. I went to pick out an expensive watercolor painting and instead I got a mirror. It was really like A Season in Hell, for both of us. But inspiration doesn't always have to be someone sending me half a dozen American Beauty roses. There's a lotta inspiration going on between the murderer and the victim. And he had me so nuts I wound up doing this nine-minute cut that transcended anything I ever did before.
Cale said in 1996 that Smith initially struck him as "someone with an incredibly volatile mouth who could handle any situation", …