Due to the nature of script kiddies (someone with limited knowledge of hacking using bots/scripts/paid services to attack a host over the internet) being able to hire a devastating "DDOS" attack on a website, a service such as Cloudflare (CF) is required to help block attacks and keep the website safe.
Most website owners use CF, the free plan more than adequate to protect from attacks. Unfortunately we wanted more protection over our website so have used the Pro plan at $20 per month for the last 18 months. Believing the extra features would benefit our site in case of problem (mainly the ability for CF to load our website from a backup) and after paying some $300+ to CF, we found that this feature did not work yesterday!
Part of this problem was that Cloudflare use their own SSL certificates (instead of ours which are valid for 540+ days), and yesterday even this multi-million dollar company displayed problems and took us down with it. For some reason loading out of date (Cloudflare SSL) certificates when trying to load our website from a backup server.
There is an option in Cloudflare to use our own SSL certificate instead of theirs, but to use this feature we'd have to pay Cloudflare $200 per month!
We could look for an alternative to CF but no doubt there would be new issues to face after switching elsewhere.
Unless we paid another £1000+ per month (plus £2500 on computer hardware) for a clone setup in a different location using completely separate everything (internet provider connected to a different phone exchange, video server providers, webservers etc) there are single points of failure beyond our control and out of our hands.
Unfortunately each aspect of this project means many possibilities for problems. Local computers, local TV signal, local internet line, local electricity, remote video servers, website programming, security updates, webserver, Cloudflare(!) etc.
With 20 minutes video downtime during the England v New Zealand game (nice timing, less than 0.003% chance of this happening, 20 minutes out of 1 year+ of uptime!?) perhaps fresh in some people's memories (a fault on the BT network nearby, resolved within 24 hours - should have been 4 hours as we pay so much), now this fault with a WordPress update (WP is used by 75,000,000 websites) not installing correctly and then Cloudflare dropping the ball when switching to backup version of the website. It might look like we are unreliable but:
Stream uptime is actually over 99.98%! Other than the 1-2 hour downtime in the last 16 months or so (20 mins mentioned +1 hour in January), the streams themselves are extremely reliable (they automatically restart every few days but this takes seconds, and some of the less popular channels need manual restart on occasion).
In January one computer had an issue but I transferred the channels to another computer within an hour or so. Talking of odds: I was out of the UK at this time. Everything had been perfect for months so I thought I could get a short break but the universe had other ideas. One computer (out of four) in the UK developed a fault, I logged in remotely to resolve it when at this very time the entire internet went down in the town I was at, no mobile signal, no internet in my hotel, no internet in bars or restaurants. This lasted for 1 hour, you calculate the odds. 1 hour out of a year uptime? Under 0.01%.....the chance of a computer breaking at home during this time? Astronomical!
And yes some of you might have been affect by our own SSL issues in May, after our 12 month anniversary our SSL certificate expired and we missed renewal notifications (email must have gone to spam and been missed), causing a few hours of website issues for some users while a new certificate was issued, but the streams were still working and we have since renewed for 2 years and moved SSL provider to the same as our domain name so everything is in one place.
So on that note:
What options are there if the website was ever offline again?
Use a VPN and connect directly to the BBC iPlayer and ITV Hub websites (All4, My5, UKTV Play, etc). We can offer discounts for vpn-uk.co.uk or you can hold on for a few weeks as VPN access will be bundled in with British TV Club memberships (this option will be easiest so you don't have to remember 2 different website logins). We already have users testing and it has been working fine for a few months. yes we mentioned this ages ago but recently got side tracked with the IPTV server.
Using an IPTV box. If it wasn't for the downtime yesterday I would have been working on the members database linking with the IPTV server. So you can log in on your IPTV box and watch our channels without going via the website. More on this very soon. In the meantime consider buying a "Formuler Z8 IPTV box" so it arrives in time when we can provide details on how to connect to our IPTV server. I highly recommend the Z8 as the developers are improving it all the time.
If you can't afford a Z8 then look on Ebay for Z7 or Z7+ or also working very well is "Dreamlink Dlite" IPTV box. Timeshift, recordings work great (requires a USB memory stick).
The VPN is the best option for resilience. If our internet line had a fault then the channels would be offline on the website and the IPTV server. The VPN server is separate, and you would get the channels direct from the BBC / ITV website etc.
Failing that, watch some stuff on Youtube! There is a lot more on there than people might realise, full length documentaries, old TV programmes, all sorts. What are your interests?
Unfortunately I don't think there is anything in life which is guaranteed to work forever and not develop faults. Cars, washing machines, computers, electricity supply, etc. Even the biggest companies can have issues!