"We've had this discussion. There are two of you involved in this, and if Rory's willing to work on any sort of relationship, that's for her to decide. Knock it off," she says.