Following on from @JanPeter’s introduction, this is a thread to discuss co-ops owning and running their own server hardware, this is something that Webarchitects and Hostsharing do.
My only thought on this at the moment is that it is capital intensive (the servers cost a lot of money) and the competition is basically insane — we are competing against one of the worlds richest corporations which can afford to make it’s own hardware including chips and which had sales of $5 billion a quarter in 2017 and, so far, has sales of $6 billion a quarter in 2018, if you want a modern example of the tendency of capital to result in a monopoly then it is hard to think of a better one than AWS.
That said, I think it is crucially important to own and control the whole stack, otherwise all your data is in someone else’s hands.
I agree. And this is a nice reminder of the bonkers monopoly power of Amazon. It’s amazing how most people still seem to think they are just a shop and often don’t think of them as a “tech giant” like Facebook and Google at all.
Incidentally, I recently found myself reading this comparison between Sonos One and Amazon Echo (no idea why as I’ve no interest in either!) and they also noted how the fact that Amazon can manufacture its own hardware and chips is what gives them the edge even though Sonos runs on the Amazon platform.
All of these include elements of scaling/compute/storage that are necessary when deploying applications these days to keep the costs down. I just wonder if there is a way we can decentralise some of the compute aspects of the stack so it doesn’t all rely on one co-op.
@nick was involved in some of these discussions and setup a OpenShift stack for testing.
However this doesn’t really answer the question of raising capital for the actual hardware itself.
Worth adding that OpenShift is in essence Kubernetes under the hood but with a nice packaging, a user interface and some additional tooling. I think given Kubernetes has more or less “won” the container orchestration space that having a managed and multi-tenant cluster that was cooperatively owned would be excellent for those who want something lower than the level of whole virtual machines.
@nick can speak to this quite a bit as he has been closer to it more recently than I. We managed to get GitLab publishing to OpenShift because it is just Kubernetes but I assume this is far further along than when I last checked in. Perhaps @matt can fill in.
Feel then as I do now that as well as simply being owned by cooperatives or a cooperative, such provision should have all the slick developer experience people would expect - leveraging something existing like Kubernetes, OpenStack or OpenShift would get it a long way - to make a transition from the regular cloud to this easier. Indeed, it should provide upgrade paths that encourage it, libraries for common tool sets (Ansible and Terraform etc) and so on.
Another one that I’ve been looking at is Flynn, which is similar to Dokku but supports scaling across a cluster of servers.
I’ve not really tried it because I’m not yet running anything requiring any more than a single modestly sized VPS but it seems promising. Not sure if anyone here has had a chance to have a good look at it.
Myself and @nick when at Outlandish did comparative analysis of all the major container based PAAS offerings including Dokku and Flynn last year at around this time.
Building on the work done at GDS, we whittled down 18 plus offerings to three or four and built prototypes in them feel them out - Flynn was one of them but we decided against it in the end. After this and much debate we ended up using Red Hat’s OpenShift. It had the right combination of robustness, features, ease of use (simple but not too simple as to have no features so you’d have to invent a persistence layer) and backing by a major vendor that wasn’t going anywhere. It is built on existing technologies (Kubernetes) rather than reinventing the wheel and has a strong community around it. We then built a three machine cluster on AWS. I’ll let others closer to the project now fill in the gaps since then.
Perhaps @matt can share the big spreadsheet we did here. Should be in the COLA section of the GDrive. I think there is justifying documentation as to why we passed on the others as well. If memory serves there was one massive missing feature in Flynn (apart from not being based on something generic) which eliminated it for us.
Just to chip in to this thread by reminding that, as far as I recall, @shaun has a couple of decent servers available, and currently sitting idle, in a cooperatively run data centre facility in Ashton.
I love the model of having a co-operative owned/managed Kubenetes/OpenShift cluster as the technology seems to have a lot of traction/tooling around it (e.g. the integration with GitLab that @kawaiipunk mentioned).
It’s not trivial to run/manage though, and needs a fair amount of resources, so it is not a casual undertaking.
As well as the hardware resources there is the learning curve for the people who will manage it and the cost of that in terms of time (or the cost of employing someone who has Kubenetes/OpenShift experience).
In addition I believe there would need to be some development work done to sort out an automatic billing system for the use of server resources, as far as I know this isn’t something that either Kubenetes or OpenShift ship with?
It is my humble opinion that the cost of the above would be in the tens of thousands of pounds and it is hard to see how this could therefore be got off the ground without a substantial quantity of capital, or am I missing something here?
Also, there are some applications that just don’t play well with containerisation so that is something to bear in mind.
I mean personally we will probably be setting up our own Dokku cloud at some point to host containers for clients so we can stack a lot of applications up on the server and improve our deployment pipelines. It would be ace to share some of this infrastructure with others co-ops.
I think we just need to start small and within CoTech rather than selling to external clients directly.
I think no one is imagining that any of these initiatives could occur without a substantial input of capital! However it is something worth exploring, even working out how much this would cost end to end and then attempting to fund it.
Kubernetes come with no billing support as it is a very generic tool but I would imagine that there might be something in the OpenShift eco-system that would assist this.
If you read the above document you’ll see one major limitation of Dokku is that it can’t operate across multiple hosts or at least could not last year when we looked into it - so a Dokku cloud is a non-starter, sadly. Which is why we eliminated it in in our process in favour of OpenShift.
I don’t mean multiple domains. I mean multiple machines in a cluster whereby the individual container is scheduled onto a host where it has the resources it requires. If this in the mix then great! - where in the documentation is it at? Seems to be some reference to scheduler plugins on their GitHub.
Hey folks, to dig this up again. Discussions are coming back… we’re still running services off of dirt-cheap VPSes from Hetzner (Truly thrifty cloud hosting - Hetzner Online GmbH) and wishing we could afford to get onto some coop owned hardware.
The costs are really orders of magnitude greater (sometimes 10x the price for an equivalent spec VPS) and I just wonder if there isn’t a way we can strategise around mutualising these costs?
Is there some intervention around variable costs (automation, operations, etc.) that we could think about (somewhat mentioned in this comment)? A lot of coops have skills in this department that could be traded.
Are the high costings for colocation purely out of fixed costs? Forgive the ignorance. I’m not for a race to the bottom but trying to see if there is some way we can find a way forward here. Inspired by https://www.org.meet.coop…
I think getting some sort of mapping of the real and ongoing costs is a useful starting point. I sort of want a table.
Feel this thread always gets mired in “it’s really expensive” without saying if we are talking 10k or 100ks or millions. How much does a real co-located piece of hardware cost in 2020?
I would think, without basis, that the needs of the cloud giants had driven down the costs quite substantially - however recognise that for both Google and Facebook, they have now entered the realms of bespoke hardware. But there aren’t the only ones in the mix and people are running private clouds in presuambly economically sustainable ways.
