husband problems

I have two daughters from different fathers, my husband is not the father of eather but is a wonderful stepdad and husband, but we have a problem... many time my youngest daughters dad cals or texts to see how she I'd my husband freaks out, he doesn't want him to get her when it's not his weekend no matter what and he says it's because my ex doesn't respect me but there are obviouse other reasons, also my husband went to jail for a month for driving under suspension and so I've been taking care of my girls all by my self with no free time, my ex offered to take both girls to the fair( which I wanted to take them to but couldn't afford to and still wanted them to b able to goto) for a couple of hours and I let him and now my husband is so angry he left our visit! He said I shouldn't care about my exs felings when I tried to explain my ex lived with both my daughters for over a year so he cared about both of them, I told him he'd want to see them both if we broke up and that wasn't even the point, I was going crazy with no free time and had a chance to breath for a few hours and I took it, how can I het my husband to see my side of things? Everytime my ex comes to get his daughter or ne thing my husband fights with me!!! I'm so sick and tired of this and it's ruening our relationship, I've understood his side but how can he understand mine? Please help!