The Modern Lovers were formed in 1970 by teenage singer, songwriter, and guitarist Jonathan Richman. In early 1971, the band's membership was settled as Richman, Jerry Harrison (keyboards), Ernie Brooks (bass) and David Robinson (drums), with Richman's friend and original band member John Felice joining them occasionally as his school commitments allowed. By the autumn of 1971, through their live performances in Boston and New York, they had begun to attract the attention of several record company A&R men, including Stuart Love at Warner Brothers, and Allan Mason and Matthew Kaufman at A&M. The band made their first recordings for Warner Brothers at the Intermedia Studios in Boston in late 1971; these included the version of "Hospital" which was later to feature on the album.
In April 1972, the Modern Lovers traveled to Los Angeles where they held two demo sessions; the first was produced for Warner Brothers by John Cale, formerly of the Velvet Underground, while the second was produced by Allan Mason and Robert Appère for A&M. Both sets of sessions yielded tracks which, although originally recorded as demos, eventually found their way onto the album. The Cale sessions produced "Roadrunner", "Astral Plane", "Old World", "Pablo Picasso", "She Cracked" and "Someone I Care About". The A&M sessions yielded "Girl Friend", "Modern World" and "Dignified and Old" (which, although not included on the original LP, was included on later CD reissues).
However, the band were initially undecided over which record company to sign for, returned to Boston, and also did some recordings organised by Kim Fowley and produced by Stuart "Dinky" Dawson. Eventually, in early 1973, they signed with Warner Brothers and agreed that John Cale should produce their debut album. In the meantime, they undertook a short residency at a hotel in Bermuda. Returning to California in the summer to work with Cale, it became apparent both that there were personality clashes between some of the band members, and that Richman now wanted to take a different approach to his songs - much more mellow and easy-paced rather than the earlier aggressive hard rock. The sessions with Cale were terminated before any new recordings were completed. Warner Brothers then engaged Kim Fowley to work with the band, but by this time Richman refused to perform some of his most popular earlier songs live. The band were also affected by the death during the sessions of their friend Gram Parsons: on the day before Parsons' death, he and Richman had played miniature golf and discussed recording together. The sessions with Fowley were aborted, although two tracks, "I'm Straight" and the original recording of "Government Center", and possibly others, were later issued on CD versions of The Modern Lovers. Warner Brothers withdrew support from the band, and, early in 1974, the original Modern Lovers split up.