I like to setup a re-run strategy, where we rerun any tests that has failed, until it fails three times in a row. If the test is still failing after two times it will be a real failing test and not something caused by network, timing, browser or other infrastructural issues.
Still you should record/log the flaky-ness of the tests as sometimes it is a really issue that users might run into. So always research your flaky tests once in a while, but with the rerun strategy you can postpone this until you have some time.
For protractor have a look at protractor-flake, which might help to re-run your tests.
Random fails will result in losing faith in test suite. Make sure you take time to come-up with a good fix. Either write it down as technical-debt or stop everything else and fix it right away.
Like other already said minimise on the end-2-end tests as they have the highest flakiness ratio.