Lauren Silverman Was 'Unhappy in Her Marriage' When She Met Simon Cowell

07/31/2013 at 08:45 PM EDT

Simon Cowell and Lauren Silverman

Getty; Sipa

Simon Cowell has yet to speak out about becoming a father but Lauren Silverman, the woman who is pregnant with his first child, is apparently happy about her relationship with The X Factor judge.

"She is ecstatic to be with Simon," a source tells PEOPLE, adding that Silverman, 36, and Cowell, 53, have been seeing each other for more than a few months.

Although Silverman is still married to a Cowell's friend Andrew Silverman, the source says she and Andrew are in the process of divorcing.

But her affair with Cowell may not have been the only factor in their split.

"It was probably not the best relationship when she first met Simon," the source says of the Silvermans' marriage. "It wasn't like they were blissfully in love at the time."

Adds the source: "She was definitely unhappy in her marriage and it's been a while that the marriage was not great. Andrew is a very nice guy but all marriages don't last."

And reports that Cowell and Andrew were best friends also seem to be overstated. "Andrew and Simon were not best friends," the source says. "They are friends. They have both been vacationing in Barbados for years."

– Reporting by Elizabeth McNeil