Feel then as I do now that as well as simply being owned by cooperatives or a cooperative, such provision should have all the slick developer experience people would expect - leveraging something existing like Kubernetes, OpenStack or OpenShift would get it a long way - to make a transition from the regular cloud to this easier. Indeed, it should provide upgrade paths that encourage it, libraries for common tool sets (Ansible and Terraform etc) and so on.
Another one that I’ve been looking at is Flynn, which is similar to Dokku but supports scaling across a cluster of servers.
I’ve not really tried it because I’m not yet running anything requiring any more than a single modestly sized VPS but it seems promising. Not sure if anyone here has had a chance to have a good look at it.
Myself and @nick when at Outlandish did comparative analysis of all the major container based PAAS offerings including Dokku and Flynn last year at around this time.
Building on the work done at GDS, we whittled down 18 plus offerings to three or four and built prototypes in them feel them out - Flynn was one of them but we decided against it in the end. After this and much debate we ended up using Red Hat’s OpenShift. It had the right combination of robustness, features, ease of use (simple but not too simple as to have no features so you’d have to invent a persistence layer) and backing by a major vendor that wasn’t going anywhere. It is built on existing technologies (Kubernetes) rather than reinventing the wheel and has a strong community around it. We then built a three machine cluster on AWS. I’ll let others closer to the project now fill in the gaps since then.
Perhaps @matt can share the big spreadsheet we did here. Should be in the COLA section of the GDrive. I think there is justifying documentation as to why we passed on the others as well. If memory serves there was one massive missing feature in Flynn (apart from not being based on something generic) which eliminated it for us.
Just to chip in to this thread by reminding that, as far as I recall, @shaun has a couple of decent servers available, and currently sitting idle, in a cooperatively run data centre facility in Ashton.
I love the model of having a co-operative owned/managed Kubenetes/OpenShift cluster as the technology seems to have a lot of traction/tooling around it (e.g. the integration with GitLab that @kawaiipunk mentioned).
It’s not trivial to run/manage though, and needs a fair amount of resources, so it is not a casual undertaking.
As well as the hardware resources there is the learning curve for the people who will manage it and the cost of that in terms of time (or the cost of employing someone who has Kubenetes/OpenShift experience).
In addition I believe there would need to be some development work done to sort out an automatic billing system for the use of server resources, as far as I know this isn’t something that either Kubenetes or OpenShift ship with?
It is my humble opinion that the cost of the above would be in the tens of thousands of pounds and it is hard to see how this could therefore be got off the ground without a substantial quantity of capital, or am I missing something here?
Also, there are some applications that just don’t play well with containerisation so that is something to bear in mind.
I mean personally we will probably be setting up our own Dokku cloud at some point to host containers for clients so we can stack a lot of applications up on the server and improve our deployment pipelines. It would be ace to share some of this infrastructure with others co-ops.
I think we just need to start small and within CoTech rather than selling to external clients directly.
I think no one is imagining that any of these initiatives could occur without a substantial input of capital! However it is something worth exploring, even working out how much this would cost end to end and then attempting to fund it.
Kubernetes come with no billing support as it is a very generic tool but I would imagine that there might be something in the OpenShift eco-system that would assist this.
If you read the above document you’ll see one major limitation of Dokku is that it can’t operate across multiple hosts or at least could not last year when we looked into it - so a Dokku cloud is a non-starter, sadly. Which is why we eliminated it in in our process in favour of OpenShift.
I don’t mean multiple domains. I mean multiple machines in a cluster whereby the individual container is scheduled onto a host where it has the resources it requires. If this in the mix then great! - where in the documentation is it at? Seems to be some reference to scheduler plugins on their GitHub.
Hey folks, to dig this up again. Discussions are coming back… we’re still running services off of dirt-cheap VPSes from Hetzner (Truly thrifty cloud hosting - Hetzner Online GmbH) and wishing we could afford to get onto some coop owned hardware.
The costs are really orders of magnitude greater (sometimes 10x the price for an equivalent spec VPS) and I just wonder if there isn’t a way we can strategise around mutualising these costs?
Is there some intervention around variable costs (automation, operations, etc.) that we could think about (somewhat mentioned in this comment)? A lot of coops have skills in this department that could be traded.
Are the high costings for colocation purely out of fixed costs? Forgive the ignorance. I’m not for a race to the bottom but trying to see if there is some way we can find a way forward here. Inspired by https://www.org.meet.coop…
I think getting some sort of mapping of the real and ongoing costs is a useful starting point. I sort of want a table.
Feel this thread always gets mired in “it’s really expensive” without saying if we are talking 10k or 100ks or millions. How much does a real co-located piece of hardware cost in 2020?
I would think, without basis, that the needs of the cloud giants had driven down the costs quite substantially - however recognise that for both Google and Facebook, they have now entered the realms of bespoke hardware. But there aren’t the only ones in the mix and people are running private clouds in presuambly economically sustainable ways.
Slighted off key and on key. There is a fundamental mistake in the operation and attraction of capital for this kind of service. Coops can not be built in the current way they are funded. I will come up with solutions and ideas. Presently the global economic outlook for the future is not bright so investments are scare from the public individuals as we head towards a great depression. I see data centres value be diminished very quickly. Since large operators like Cisco try to compete. There will be alot of competition eroding alot of smaller data centres or making them be purchased from largers giants. Though coops have a huge advstange in this arena I simply need to develop my cooperative model and its article to file at a regulator that wont swipe us away from the fold of commercialisation and is good to attract capital from so anyone no of any jurisdictions that have good laws but fewer regulations some but flexi. I welcome feedback
We recently brought a new file server, it isn’t racked yet, due to the lack of capital we opted for the cheapest, minimal options for a machine that could replace our existing server and also give us a serious increase in disk io, reduce the electricity demand and also some additional space, it is a 2U server with 20 2.5" disk bays, 8 populated with 1.9TB SSDs (so there is room for more when we have the funds) and this cost about £7.5k.
The next front facing server we are looking at getting is a AMD 2nd generation EPYC 24 core CPU (48 with hyper threading) and 512GB of RAM and from memory this will cost in the region of £4-5k.
I know it depends on their needs, but roughly how many sites would you expect to host on that machine?
Obviously the second rung is complete cost of ownership here, with includes paying for someone to maintain it.
Not asking in the abstract here. I’d genuinely like to consider whether a few coops should do this, pertaining to this other thread on a plausible Heroku alternative.
