New relationship & social services got in touch

[Boopywhoopy]

Hello
I met someone around 5 months ago. Everything was going great until his ex wife started trying to split us up.
She had him arrested and made false accusations about him. She contacted Claire's Law on my behalf, not to protect me but to cause harm.
I knew everything before I saw the domestic abuse officer that day. She then said that she would have to contact social services. I have 2 beautiful children and love being a mum.
Social services got in touch the next day and basically risk me I had to leave him or I wouldn't be prioritizing my kids. He is an amazing dad. Has no history of child abuse and I have proof that he was the abused and also how much of an amazing dad he is.
My question is, can they really control us like this? He doesn't even have a criminal record as the statement was retracted and bail conditions lifted.
How does he live a normal life and have a relationship if he is being controlled this way?
I have chosen not to get the social services involved. But I wonder, if was to, how would that be and what would I expect?
I don't want to end this relationship..
I have kept him away from the kids to be safe. And I am happy to do this if it means that we can still be together but he has huge anxiety and is saying how he thinks he needs to end us now to protect us. My whole life has been turned upside down. And the kids miss him..
Any advice would be greatfully appreciated, thanks