Though ‘It Ends With Us’ takes place in Boston, it was actually largely filmed in New Jersey

Colleen Hoover’s book It Ends With Us comes to life with the recent movie adaptation, but some iconic locations may look a bit different on screen.

Starring Blake Lively and Justin Baldoni, the movie follows a young woman named Lily Bloom who moves to a new city and eventually meets and falls in love with a man named Ryle. However, their relationship eventually becomes toxic as Ryle becomes physically and emotionally abusive.

Among various book-to-movie changes, the filming locations also strayed from the source material. While the book and movie are both set in Boston, filming largely took place in New Jersey.

Ahead of the release, PEOPLE spoke exclusively with one of the film’s location managers, Jeff Brown, who spoke extensively about the biggest settings from the book and how they were adapted for the screen.

On why the filmmakers decided on New Jersey as the backdrop instead of Boston, he noted that the decision “was primarily a financial decision,” but the crew worked hard to give the Garden State an authentic New England feel.

“I think Justin had really fallen in love with a lot of the looks of Jersey as Boston and we were really trying to keep his vision of what he had seen alive,” he said of Baldoni, who also served as the director and producer for the project. “We all knew from the book and from the script that Boston was a major part of the story. So with locations, sometimes you’re looking for things that are underneath the story in a way. What sells Boston that isn’t necessarily Boston?”

“It’s about the people more than the location … it was finding places that Justin and Blake would be comfortable performing these scenes,” he adds. “Their faces do not necessarily need to say Boston, they say Ryle and Lily, and that’s more important than the specific of some street.”

Read ahead as Jeff Brown breaks down the biggest filming locations from the movie.

Warning: spoilers for It Ends With Us ahead.

Related: It Ends with Us Cast Drama Explained: What Is Going on with Blake Lively and Justin Baldoni?

Lily’s childhood home

Alex Neustaedter and Isabela Ferrer in It Ends With Us
For Lily’s childhood home, Brown says he worked on finding a place that was multifaceted as it had to “exist both as when Lily was a young woman” and when she comes home to visit her mother. “You have to find a house that satisfies both of those narratives,” he explains.

To capture that essence, Brown settled on a 100-year-old home located in Plainfield, New Jersey, which doubled as Maine for the movie. “Plainfield has a couple of streets that [are] very old. The [houses] are all very big and very nice. And the homeowners had preserved a lot of the original moldings,” Brown notes. “Those are things that you’re looking for as a location scout, you’re looking for those details because they’re things that are really hard to replicate.”

“The structure of the house is one thing, and then the art department would really bring in the character of Lily’s mother and Lily’s room,” he adds. “The room itself sells the character and that’s a big part of it.”

Ryle’s loft

Surprisingly, Ryle’s loft was actually created in a building owned by Veris Residential in Weehawken, New Jersey. “The loft was something that was a very tricky, difficult thing to find,” Brown notes. “Basically, the team created his loft within a common space of a rental building. So all of that stuff, the bedroom and the kitchen, all that is fabricated. None of it was real.”

“If you’re shooting in a location for over two weeks, you can’t have a really small location. You’re going to have to have something that’s pretty big because there’s how many camera angles that we have to allow,” he explains. “Justin did not want to film on a stage, so there was nothing built on a stage. These were all practical locations.”

Lily’s flower shop

Of course, one of the most central locations from the book and movie is Lily’s flower shop, which Brown says was filmed at a “vacant storefront in Hoboken,” New Jersey. For that filming location, he notes that they really wanted to play into the “distinct old-city vibe” that both Hoboken and Boston possess.

“Other cities don’t quite have the Revolutionary War as part of their history,” he adds. “So the fact that these cities in both Jersey and in Massachusetts, they go back that old, that’s something that I think plays into the story very much.”

He adds the location really came to life thanks to collaboration with the art team, including featuring flowers from local florists. “There’s a very specific vibe to what she’s doing in the shop,” he notes. “The art department created the evolution of the shop through the set dressing and through the props. It’s a very tactile kind of location and then where they’re rebuilding this within an old space, it’s neat that they could pull something off like that.”

The rooftop where Lily and Ryle meet

Unlike other filming locations, the iconic rooftop scene where Lily and Ryle first meet was actually filmed in Los Angeles. Brown notes that they weref going to use the rooftop where they filmed Ryle’s loft scenes, but weather ended up playing a big factor.

“When we were doing the filming, it was January in New Jersey and we had had a pretty cold winter,” he notes. “So trying to do dramatic scenes on a freezing rooftop on the Hudson River, it’s going to be really difficult for Justin and Blake. An executive decision was made, when we were doing some of the prep for this round of filming that the East Coast was going to be too frigid for the filming, so they brought that to Los Angeles.”

He adds that for the rooftop scene, Baldoni gave him a specific reference of a rooftop scene from Her starring Joaquin Phoenix. “Justin liked that vibe of the rooftop from that and was trying to find something like it,” he says. “[The location] had the kind of vibes he was going for. It’s a beautiful movie, so we were trying to capture that sensibility.”

Lily and Ryle’s big kiss

Lily and Ryle’s big kiss from the movie takes place on Newark Avenue in Jersey City. “It’s a commercial stretch and we shot after Christmas, so we worked with them to take down some of the Christmas decorations, but leave up the string lights because our director of photography really responded to those as a texture in the background, especially when you have the romantic scene of the first kiss,” he explains.

“We really needed those types of visual fireworks to show what is going on to the characters inside. It helps the mood of the piece, as well, that that’s a very romantic street.”

Atlas’ restaurant

The setting for Atlas’ restaurant Root is actually a real place you can dine in! Though, it’s really called South House, a fine dining restaurant that serves Southern dishes as well as cocktails in a rustic setting.

The restaurant is also located on Newark Ave. Brown notes that the interior and exterior of Atlas’ restaurant was filmed at the location.

The farmer’s market where Lily and Atlas reconnect

In the final scene of the movie, Atlas and Lily cross paths again at a local farmer’s market, which Brown notes is close to the location for Atlas’ restaurant. “It’s called Van Vorst Park and it’s a public park that’s in a very old section of Jersey City,” he explains.

“They get approached for filming quite a bit and I don’t think they allow a lot of film shoots, so we were very, very fortunate to film there. That was another one where Justin just fell in love with Van Vorst Park and he was very much like, ‘This is where the farmer’s market is going to be.’”

Though there’s an actual farmer’s market in Van Vorst Park, Brown says they staged one in a different location because it played better with the lighting. He adds it was a group effort as they brought in local vendors from the area to create the mood.

“Van Vorst Park, they have an organization that manages it, and they turned the fountain on for us because they had closed it off for the winter,” he adds. “They were really willing to play with us.”