By the time the body of the stone dealer was discovered in his Upper West Side apartment, it had been decomposing for several weeks. The summer of 1969 was a scorcher, and the smell was so repellent that city officials ordered the building’s superintendent, Stanley, to dispose of everything in the unit. Stanley thought he might make some money off the man’s antique furniture, so he called Wally Gibbs, an antiques dealer, who called his friend and peer Vito Giallo for an appraisal.

Mr. Giallo, now 91, told this story during an interview at his home in Brooklyn.

Here is what he found in the dead man’s apartment: a table, a dresser, a desk, a trunk, a chair missing an arm, a stack of uncashed checks and no will. The city had tried to find the deceased’s relatives but came up short. There were no heirs.

At first, Mr. Giallo doubted any of the items would sell at his tony antiques store on the Upper East Side. But when he opened the small trunk, he gasped. Inside, he said, there were thousands of gems — amethysts, rubies, emeralds, topaz, opals, moonstones — nearly every kind of precious and semiprecious stone aside from diamonds, in turn-of-the-century cuts. They ranged in size from “pinhead to thumbnail,” he said, and were organized in hundreds of disintegrating envelopes. It wasn’t until Mr. Giallo examined the stack of checks that he realized the stones had been the man’s livelihood.

“I know absolutely nothing about him,” Mr. Giallo, said. “Absolutely nothing. I don’t even know if he had a business open to the public.” The super didn’t believe the stones were real, so he told Mr. Giallo that he could keep them. Mr. Giallo could not believe his luck; he had simply been in the right place at the right time and insisted on splitting any earnings with Stanley.

He sold a handful of stones to choice customers, including Andy Warhol. But for the last five decades, most of the gems have been in a storage unit. During the pandemic, Mr. Giallo’s great-grand-niece Tara Marrale, a founder of a sustainability start-up called Grooted, suggested he put them to use. They decided to approach an established jeweler to collaborate on a line of cocktail rings, necklaces and earrings.

“I must have looked at 20 or so jewelers,” Ms. Marrale, 33, said, before reaching out to Catbird, a Brooklyn brand with stores in Williamsburg and SoHo that uses recycled stones in its designs. She sent an email to the company and explained that Mr. Giallo had been an antiques dealer “to the who’s who,” including Elton John, André Leon Talley, Greta Garbo and Mark Rothko, and that he “put on Andy Warhol’s first show in 1954.”

“This might be a totally far-fetched idea,” Ms. Marrale wrote, hedging in case Catbird’s team didn’t take kindly to the pitch. Obviously, they did.

Mr. Giallo has regaled the brand’s executives with stories about the midcentury art world in New York. He even gave Leigh Batnick Plessner, the creative director of Catbird, a copy of his unpublished memoir, which traces his serendipitous path to Madison Avenue.

“His New York, and he says this in his memoir, is a city vanished,” Ms. Plessner said. That is, a city where nobody cared about being world famous for 15 minutes — at least not yet.

How Vito Met Andy

Mr. Giallo grew up in Brewster, N.Y. His father, he said, was a bootlegger who operated 19 stills hidden in various associates’ mansions. “We had to be very careful on the phone. It was always tapped,” Mr. Giallo said. “We couldn’t say where he was. When he was ready to come home, he would always say, ‘Put the water on to boil.’” When Vito was 12, his father taped several $100 bills to his son’s body and instructed him to deliver them to a relative via train. The F.B.I. was watching the house, his father told him.

At 19, Mr. Giallo moved to New York to attend the Franklin School of Professional Arts, having won a scholarship through an illustration contest. On the first day, he met Ebby Weaver, who would become his life partner for the next 51 years.

After graduation, he illustrated for “Mad Men”-style ad agencies such as Young & Rubicam and J. Walter Thompson. In 1954, at the age of 23, Mr. Giallo opened the Loft Gallery at the studio of the graphic designer Jack Wolfgang Beck. “By word of mouth, it leaked that we were looking for artists and they came to us,” Mr. Giallo said. A friend named Nathan Gluck connected the gallery to a young artist named Andy Warhol, who contributed a collection of origami to the inaugural group exhibition.

“It was one of Andy’s first shows,” Mr. Giallo said. Mr. Warhol’s origami wouldn’t stay affixed to the wall, with pieces dropping like snowflakes every few minutes. Finally, Mr. Giallo recalled, the artist said, “‘Just leave it on the floor and let people walk around them.’ So that’s what we did. He always had a solution, a simple solution to every problem.”

Within a year, Mr. Warhol invited Mr. Giallo to be his first paid assistant, working out of his railroad apartment on East 34th Street, which the artist shared with his mother. (Mr. Giallo lived a block away, in a two-bedroom apartment he and Mr. Weaver rented from Bert Carpenter, an art professor at Columbia, for $85 a month.) Mr. Warhol and Mr. Giallo collaborated on much of Warhol’s early work — except, he said, for “the shoe drawings.”

While Mr. Warhol sketched using a pad on his knee in front of the television, Mr. Giallo used a light box to produce blotted line drawings. Mr. Warhol’s mother did the lettering and prepared lunches of sandwiches and Campbell’s tomato soup.

At night, they went out. “I didn’t care for the wild parties,” Mr. Giallo said. “But Andy did. Everybody wanted to meet him. But when they did meet him, they were very disappointed because he wouldn’t say anything. He hardly ever talked.” Mr. Giallo preferred jazz and quartet shows — Ella Fitzgerald, Billie Holiday, Ray Charles, Miles Davis — which cost a few dollars per ticket. It was easy, too, to sneak in, though Mr. Giallo swears he never tried.

In the ’50s, he said, people could enjoy the best meal of their lives for $1.85 — the equivalent of roughly $22 today — and walk into Lord & Taylor asking for a reference for “a lighting guy,” only to be connected with Alexander Calder’s best friend. “It was a lot more fun,” Mr. Giallo said. “You could sleep in Central Park without worrying that you were in danger. A lot of people did that on very hot days.”

Mr. Giallo stopped working for Mr. Warhol in 1957, after the two had a falling out. (Mr. Gluck, his replacement, became Mr. Warhol’s most significant assistant.) In 1961, Mr. Giallo opened his first antiques shop, on Third Avenue, where he drew a clientele of Abstract Expressionist painters such as Mark Rothko, Franz Kline and Lee Krasner. “When they bought something, I delivered it to Rothko’s apartment, which was just around the corner,” Mr. Giallo said.

Mary Gabriel, an art historian, noted that those artists often welcomed strangers into their work spaces. “If you wanted to see an artist’s work, you would go to their studio, and you would talk to them,” she said, adding that today, “It pains me to think that people look at a 10-foot painting on the internet.”

Decades later, after Mr. Giallo moved his store to Madison Avenue, Mr. Warhol popped in and said, “‘Wow, you must be on Easy Street.’ He just loved the idea I was on Madison Avenue. And then he came every single day for the last eight years of his life.”

After they reconciled, Mr. Giallo told Mr. Warhol about the trunk of gems, and offered to sell some for the Pop artist’s upcoming foray into jewelry design. “He looked at one box of 35 to 40 stones and said, ‘I’ll take them all.’” But Mr. Warhol died of complications from surgery before he was able to use them, and the gems sold at his estate sale in 1988 for about $10,000, according to Sotheby’s.

‘You Have to Be Very Curious About Things’

It’s fitting that Warhol’s gems ended up at a Sotheby’s estate sale; for most of his career, Mr. Giallo bought his wares at auction there every other Sunday. He was such a frequent buyer that, he said, Sotheby’s guards would lend him their keys to unlock the display cases so he could closely examine items like Art Deco jewelry boxes and porcelain tableware.

“Vito was a pro in the $50 to $5,000 range,” said Kevin Tierney, the Sotheby’s silver specialist who joined the house in 1964. He described Mr. Giallo as “a charming person with a keen eye for quality and style, quick to offer a friendly discount to young employees at Sotheby’s.”

Mr. Giallo described his buying strategy as instinctual. “When I see something that I’m interested in, it’s some sort of vibration that you feel with the item. It’s hard to explain,” he said. “You wonder where it’s been, who had it, who owned it, why they had it. All that, you just wonder about. I like that feeling. I get a little thrill touching something that’s two or three hundred years old.”

Often, the appraisers at Sotheby’s would underestimate the value of an item, and Mr. Giallo would buy it for a song. “I bought so many chandeliers as glass, and they were rock crystal,” he said. “They made big, big mistakes on rock crystal.” Once, he bought a silver whiskey glass for $100 that turned out to be solid gold. “A lot of things slip by at auction. That’s one reason I went every two weeks.”

He and Mr. Weaver, an interior designer, would also take weekend trips to sales outside the city. At a roadside shop, he once reached into a bargain barrel and withdrew a Chinese drinking vessel made of pottery from the Han dynasty, which he later sold for $1,500.

“When you get a few of those, you get very excited about antiques,” he said. “You want to find that one treasure.” His biggest coup came at a small estate sale in Connecticut, where he paid $12 for a watercolor painting of a pastoral dance scene, which turned out to be a Charles Demuth. (It sold for $180,000 at auction.)

“An object, I think, that is worth buying, it somehow talks to me,” Mr. Giallo said. “You have to be very curious about things.”

Nowhere is that trait more apparent than on the walls and surfaces of Mr. Giallo’s apartment. It is easy to feel overwhelmed at first by the volume of blink-and-you-miss-it marvels, including a narwhal tusk, a rectangular meteorite chunk and Italian paintings from the 18th century, as well as works by Warhol, Matisse and Picasso.

Ms. Marrale, Mr. Giallo’s grandniece by marriage who helped broker the Catbird collaboration, and her husband, Jonathan Marrale, live in an apartment upstairs. She has been helping Mr. Giallo organize his various collections. “You think you’re going to go downstairs for five minutes,” Mr. Marrale said. “But he’s so interesting that you’re there for two hours, and it feels like no time has passed.”

Before the pandemic, Mr. Giallo would spend every morning at the cafe across the street. He’d order coffee and a pastry, sit down, and start talking to a stranger. “It was a thing for him,” Ms. Marrale said. “He really digs deep.”

Ms. Plessner, of Catbird, has enjoyed asking Mr. Giallo about her art heroes, like Frank O’Hara. Mr. Giallo knew him in passing, “but there was another poet who I was very dear friends with,” he said. “Have you heard of John Ashbery?”

Despite his many brushes with fame, Mr. Giallo considers himself an introvert. When Ms. Marrale first suggested the jewelry line, he replied, “Why would people be interested?’”

“Because you have an amazing story,” she told him. “Like, this is not like a normal person’s life.”