Hi Jamie, once again this would be valid Product Feedback - I will raise it with our engineers.
Regarding the removal of cross-filtering once it is enabled: indeed, this can only be done from the filter bar. As for disabling the option to use a certain insight to cross-filter, it is also not possible right now.
Thanks for your insights here!