Both are options that I’m personally not too fond of, but can nonetheless respect that these probably fit the threat model of some people. Signal is not self hostable, not all that private (still better than whatsapp atleast), and it’s servers are located in the States. Does have some FOSS clients like molly atleast.
Matrix is a bit more complicated in my opinion. While the protocol itself is fine, the history behind it is what makes me not want to use it personally. Namely, it being a project that emerged from Amdocs, an Israeli telecommunications company previously suspected of eavesdropping and spying on people. While open-source and self-hostable, it has a network effect going on where everyone registers on the main instance and self-hosting is apparently incredibly resource intensive. Plus, most instances require you to give an email address.
Personally, I’d recommend looking into GNU Jami and XMPP. XMPP may not have encryption as a built-in component of the protocol in the way that it is with Matrix, it nonetheless exists as an option and there’s plenty of no-email-required instances. XMPP is also less resource intensive to self host should you wish to do that.
Jami doesn’t work with servers but is instead distributed and peer-to-peer along with being libre software. Only knock against this one is that the desktop client is based on electron, which isn’t a problem for me personally but there are people who’re nonetheless opinionated about that.
Maybe depends on which server implementation you use, I could be out of date. I heard about Synapse, the “flagship” implementation, being heavier though you of course have the option to use one of the lighter weight implementations. Personally, I don’t self host and I am mostly going on second-hand information that I’ve read from people who do self host.