Serverless Flarum

Franz

SteveSmith I've been wanting to answer this for a while. Now I finally have the time...

Sorry if this came across as simply brushing away your comment. Without having much experience (apart from reading a few articles here and there), I can't say much about the technical merit of the "serverless" technology. As far as Flarum is concerned, I don't see much use for it (but feel free to prove me wrong). Going down the classical PHP hosting route was a very early and fundamental decision for the architecture of Flarum, and that is clearly a market deserving to be served. If this doesn't stop us from being compatible to the serverless approach, then so be it. We are not, however, going to jump on that bandwagon (which, to me, is clearly a hype, in the Gartner Hype Cycle sense of the word) just because everybody does it. Again, fully adhering to 12-factor principles already gives me a lot of benefits in line with many modern development principles. And just because "traditional" PHP hosting is old, doesn't make it merit-less. (Most importantly, the deployment story of traditional PHP hosting is still unbeaten for non-tech-savyy people IMO.)

For GraphQL, a lot has been said, and there have been many good articles comparing REST-ful APIs (which we are attempting to some degree) and GraphQL APIs. Both have their strong points, and their weaknesses. We can discuss how these may or may not apply to Flarum, but "somebody else is using it" to me does not mark a technical argument based on strenghts and weaknesses. ?

Thanks for engaging the community, and happy Easter! ?

SteveSmith

Franz Hi Franz, I understand that you're busy. You better be, I'm waiting for the 1.0 version of your product. ?

Just so you know, we are in the same boat, I have a full time job and I'm trying to build a website by myself, which as you know already, requires lots of effort.

As far as serverless goes, you seem to be having some misconceptions about my arguments. I don't think I've framed my argument as - everybody's doing it, so we should too. My argument is rather that the concept offers certain advantages which outweigh the cons compared to a traditional server-based architecture. That advantage is simplicity and automation. Tell me, why would you want to do any of the following things?

Setup hosting servers manually
Setup web server on the host server manually
Setup load balancing manually
Setup backup servers manually

None of those things are fun for developers. We used to do them because we had no choice. All of those things, and many more including security, are now completely automated from the perspective of the developer. If there is no server, then there are no chores for you to do that are usually related with servers. Period. This is an obvious and giant advantage, which no 12 factor point list is going to overcome.

Now put yourself in my shoes. Let's say that I finally finish my serverless website and I'm enjoying myself since I don't have to take care of the infrastructure my website is running on. Next, I decide to add a forum, and I choose Flarum. Do you honestly think I'll be happy if I have to provision my own server just to run Flarum? Why would I do that when its clearly possible to NOT have to bother with servers?

Now, from your perspective, in the short run it would be a pain to transfer the existing Flarum code to serverless or build a serverless version. But in the long run, it would certainly be beneficial.

Also, AWS supports serverless PHP, so its not a choice between serverless and PHP. You can do both at the same time. As far as GraphQL goes, its pretty much an automated version of the standard REST API which makes it more flexible. No need to modify your API when you decide to add new service calls. But then again, you don't need to use GraphQL to do serverless architecture, but they go hand in hand, so its a good idea.

In any case, I'm here to give suggestions and not make demands, hope you choose the best for your project.

JoshyPHP

The "serverless" concept isn't new. 12 years ago it was called SQL on Rails.

Franz

JoshyPHP From what I understand, it also seems to be pretty similar to PHP's old shared-nothing process model. But I might be way off the charts here. ?

SteveSmith

JoshyPHP I don't see any resemblance to the serverless concept. And if you google this thing, you'll see that its an April Fool's joke.

Furthermore, assuming that it actually had anything to do with serverless, you argument still doesn't hold water. Just because something similar existed before doesn't somehow negate the progress the new concept has to offer. You could also say that Flarum is nothing new, 12 years ago it was called BBS Forums.

See how that makes no sense whatsoever?

Franz

SteveSmith Thanks for getting back.

Can you outline the steps we would need to take to make Flarum compatible with a serverless approach? If it means self-hosting Flarum isn't possible anymore, then I don't think it's going to happen. ?

JoshyPHP

SteveSmith I can't tell them apart. I have read the pitch from the website you linked and I understand the general idea behind the framework misleadingly named "Serverless" but if there wasn't a public repository for it I would believe it's a similar prank.

The Serverless manifesto reads a lot like a sales pitch for cloud hosting. A big selling point is that "as a developer you don’t have to think about those servers. You just focus on code." What does that even mean? PHP programmers write their software without spending a single moment thinking about the servers it will run on.

Maybe it's good for people who are developing in JavaScript and are having troubles deploying their application but I don't see any tangible benefit compared to the average PHP application.

SteveSmith

Well, I don’t know the architecture of Flarum so I can’t say what steps exactly but what I can do is tell you how its usually done. Please observe the diagram.

https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcS1oeknjwKVKnUSVm5z2hcctIDNcLfjM6RGRk20-3anCIZfpV0VTB25GafrLA

First things first, you place your static content, and javascript in an Amazon S3 bucket, which is simply a key-value store. This is where you actually host your forum, by simply enabling the web hosting option.
You then enable the API Gateway which will be used to communicate with the backend. Meaning, any clicks, posts, edits on the forum will be channeled here.
Your PHP code goes into AWS Lambdas. These are nothing other than functions you can execute on demand. Yes, you actually write PHP code, publish it and invoke it as a service. This is why its called FaaS, short for Function as a Service. These will be invoked by your API Gateway.
Any Lambda invocations, which represent data in posts, user info, etc… will go into the DynamoDB, which is a document store. Instead of having a relational database, you simply store your posts as documents.

Everything from then on is provisioned for you, security, auto scaling of servers, distributed lambda execution with respect to user count, etc. Furthermore, if you want to include GraphQL, then AWS actually offers a managed GraphQL + Apollo service called AWS Appsync. Have a look at the following video. And let me know whether this looks promising to you. I believe it simplifies development and the interaction between frontend and backend a lot.

And just for the record, I do not think you should proceed with this approach any time soon, unless you have a dedicated team of people to do it. My initial suggestion was to just let you guys know where things are moving to and what you could do in the future. I see no reason why, if your dev team grows large enough, you wouldn’t cater to the serverless community as well and simply fork Flarum so that you have a serverless version too.

Franz

SteveSmith Your PHP code goes into AWS Lambdas. These are nothing other than functions you can execute on demand. Yes, you actually write PHP code, publish it and invoke it as a service. This is why its called FaaS, short for Function as a Service. These will be invoked by your API Gateway.

Honest question: What is the conceptual difference to having a bunch of PHP files (instead of a single front controller) on a public web directory? The "on-demand execution" would then just be a HTTP request to exactly this file...

SteveSmith And let me know whether this looks promising to you. I believe it simplifies development and the interaction between frontend and backend a lot.

Well... from my current level of ignorance, it looks to me as though the "traditional" way of building PHP applications means anybody can deploy it on any type of server, and tune it to their wishes. (Yes, making things scalable requires extra effort and knowledge - but I doubt that that can be avoided when going serverless.) With serverless, it looks like you're locked in to a specific cloud ecosystem?

SteveSmith

JoshyPHP Then you have a bunch of bad developers. If you don't think where your code will run, how can you write code in the first place? How do you write a function which allocates 64 GB of ram, when you only have available 32GB. Please, let me know, I would really like to know how you pull that off.

Franz

Please keep this discussion to the benefits / feasability of making Flarum compatible with the serverless approach.

SteveSmith

Franz Well yeah, but its obviously more than that when you go full serverless, right? If you only focus on this one part, meaning you have a remote place where you make function calls, then you are still better off with lambdas because there is no server to manage. You publish and that's it.

As for being locked in, well you could use some framework to setup your infrastructure, like Serverless Framework which allows you to provision resources on different cloud services. Sure, some features are specific to certain services, but that's a small price to pay for the flexibility that you gain by sticking to one. And yes, there really is nothing to do with the servers when you start using lambdas. You literally write code and publish it. The rest is handled by the cloud provider.

https://aws.amazon.com/lambda/

So, your biggest concern then might be that you would be locked into a specific provider. I understand that and its a valid concern. But then again, if you do decide to go serverless, then you just have to pick one of the providers, and once you do, there's no reason not to stick with it. It's not like you'll be moving every 5 months to a different one and its not like Microsoft or Amazon are going anywhere soon.

luceos

Making flarum serverless will close it down to only a limited number of people. Even worse than our decision to go with composer.

I feel a lot of effort is placed in discussing something which has no merit to flarum in the first place.

SteveSmith

luceos Which is why I said that this should be a different version of Flarum, which you could work on, once you have enough resources. And its not like this is not going to be the future. The cloud is already here and its not going anywhere, its just going to become bigger.

Do you think there is ANY, and I do mean ANY chance of the IT moving back to manually managed servers? That's like saying that regular TV stations are gonna come back into fashion and companies like Netflix are gonna fade away, or that music services like Spotify will fall out of fashion and we will go back to buying CDs and. This is not going to happen.

So my suggestion is nothing else but pointing out to you guys where the future lies. And if you do decide to go down this route, you will have 2 version of Flarum, and in the not so distant future, as technology moves forward, one will be scrapped in favor of the other.

Guess which one is going to be abandoned?

Franz

SteveSmith That's like saying that regular TV stations are gonna come back into fashion

I do still see a lot of these around, don't you?

couldn't resist, sorry ?

jordanjay29

I'd like to add that Flarum is licensed under the MIT license, which makes it possible for someone to fork Flarum and make it compatible with whatever architecture they prefer. While I'm not trying to dismiss this discussion, far from it, it doesn't look like this is a direction that Flarum is inclined to head down considering two of our core developers have weighed in already. That doesn't mean it isn't possible at all, it just may mean that someone else, not our core devs, will need to develop this ability and bring it forth.

Honestly, I've seen a version of this with Sandstorm.io, but I'm not a big fan of it for dynamic, public-facing software. Sites that can be updated periodically by a single person or single group, like a blog, work better in this kind of situation than a community site like a forum. But that's just my take on it, I'm not against someone pursuing this for the benefit of those who prefer serverless architecture.

SteveSmith

jordanjay29 Which again, is why I said that if your team grows, you should consider it. I didn't say that all the hard work which was done on this forum should be scrapped and that the serverless version should be started from scratch. Forking it to a parallel version, some time in the future is the way to go in my opinion.

Franz Yup, that's a real thing and so is vinyl resurgence and I should know, since I'm an audiophile. ? But in both cases, the trend, that is the general trend of where things are headed, is obvious.

I hope I gave you guys something to think about.

Sanguine

SteveSmith Thanks for sparking this interesting discussion!

I like the abstraction from a developers perspective, but locking yourself into the power of the worlds biggest company (Amazon) feels abhorrent to me. If only there was a cross industry standardized interface for lambda execution...

Now I see why companies are moving their business apps en masse to aws (simple cost calculation) but open source developers are probably more concerned about ideals.

JoshyPHP

Sanguine open source developers are probably more concerned about ideals.

I think it's the other way around. The idea of an infinitely scalable web application that spins up server instances to match the current load is a pie in the sky. Most forums can run on a $5 / month cheapo hosting plan, their database doesn't exceed 1 GB and their biggest table will hardly reach a million rows.

This whole discussion sounds foreign to me because it's made up of highly hypothetical scenarios that don't seem to apply to anything related to web applications in general and Flarum in particular. I think there is an inverse relation between one's enthusiasm for something like the Serverless framework and the amount of code one produces.

SteveSmith

JoshyPHP Stories and assumptions aside, the serverless concept is being accepted with open arms by large corporations. That does not mean that it's the right way, but it points to where the future of development and hosting is.

I'm ready to accept that future, are you?

As far as Flarum goes, I think I've repeated myself many times and therefore, this will be the last time. Flarum might not necessarily gain on performance if it was implemented in the serverless fashion, but if you already have a large infrastructure, with many websites and databases, all built serverless, you will probably not go back to the server ways of doing things just to accommodate the use of Flarum.

My suggestion mostly has to do with what the future brings and that is cloud and serverless. If Flarum does not prepare for the future, that is pretty much just around the corner, it might be dead on arrival, and I'm willing to bet nobody here wants that.

This was a word to the wise, peace.

JoshyPHP

SteveSmith I'm sorry but it's all empty rhetoric. I still don't know what practical benefits there would be to Flarum.

Here's what it sounds like to me: the future is clearly in quantum computing so let's run Flarum on quantum computers. Sure, right now there aren't too many of them but you don't want to live in the past, do you?

« Previous Page Next Page »