After further investigation, we have narrowed this down to just one domain that's affected by this.
We even created a new random domain and that works as expected (smart static files processing can be ticked and it stays ticked after clicking apply).
Any ideas why it's just one domain that's affected by this issue ?