Intro Nowadays, most modern applications are based on a microservice architecture to achieve high flexibility, especially when thinking from a cloud-native development perspective. This approach reduces both the dependencies and the complexity of the service itself. But it increases organizational complexity and requires good communication, alignment and documentation within the development organization and beyond.
When starting a project in a microservice setup, you are often confronted with the same basic questions over and over again.
Which services and frameworks should I choose for authorization, session handling, reverse proxy, database and web application? How are the DEV and PROD environments set up locally and later in the cloud? How do I dockerize the applications? Do I use http or https on localhost? And of course: Where and how can I host my services easily and cost-effectively? This list is far from exhaustive and will become more specific and obviously more technical as the project progresses (which I really appreciate).
We at mogenius have asked ourselves these questions many times and decided to share with you how we approach them. We're using a full-featured demo app to walk you through these questions in a structured way, and perhaps inspire you to start working on your next microservices project.
So without further ado, let's get started!
The app and its features We will create a Twitter-like application with Markdown posts. This approach covers all the above issues and more.
Here's what it should look like:
To get a better idea of what we'll be talking about [|| Code 👨💻], here's the feature list of our Twitter clone:
Login with Github List posts Show a post Create a new post (with markdown) View a user profile Search through posts Show a list of trending hashtags and users This sounds pretty straightforward. To develop an approach to building these functions, we will make some basic assumptions from a higher-level perspective:
We want to separate the frontend from the backend. We need to store our persistent data somewhere. We may need to store temporary data in between with shorter data lifetimes. We will have multiple microservices, probably based on different technologies. We need a gateway to reach our microservices through one domain. We need to consider security and authorization since we have user-related data. Let's not make it too complicated and just start creating our first services.
The Approach: develop service by service Let's get to the heart of things. Based on the holistic view in the last chapter, we will merge the feature list with our higher-level requirements to create the service architecture.
What is a service in our context? In my eyes, a service is a specific problem solver. That's it. Nothing more, nothing less. And that's exactly how we will proceed - one problem, one service.
(Side note: I think there are many really good explanations of what a service is or does or should be. Feel free to make up your own mind...)
That is, we simply define each service in terms of the problem it is intended to solve. Ultimately, each of the above problems should be solved by a particular service. When you do this yourself, make sure you really stick to this approach - one problem, one service. Later on, it often happens that a service becomes a multi-purpose service, and that's not always bad - but in the beginning, you should avoid that so that your application is easy to understand and maintain. A good concept to structure this in your head is MECE (mutually exclusive, collectively exhaustive).
Following this approach, we come to the following list of services:
Services:
Frontend service Reverse proxy service Database service Session/Cache service Authorization service User service Explore service Pre-parse service We will go into more detail later about what exactly each service does.
As an overview, here is a sketch of our service architecture:
Creating an environment to start building We are getting closer to the implementation level, but we are not there yet. First, we need to make some decisions about building our infrastructure.
Cloud platform
First, we are looking for a platform that can easily deploy our microservice architecture. We want to have a simple setup and a scalable infrastructure under the hood. At the same time, of course, it should be cost-efficient. Well, you are reading this on the mogenius blog, so the choice is obvious. We're going to use mogenius to deploy our microservice architecture and we can use the free plan to get started.
However, our setup is not specific to mogenius. You should be able to run it anywhere, even on your local system.
If you don't have an account for mogenius yet, you can create one here studio.mogenius.com .
We will use Docker for containerization. This means that we will add a Dockerfile to every repository we create. Please check out the references and best practices on the Docker website.
Each service will be a repository. We host our repositories on Github because mogenius has a native integration for Github to easily connect the accounts.
And to run our containers more conveniently on localhost, we use makefiles .
Languages and frameworks Based on the services, we will discuss what language and framework we will use.
Disclaimer: When it comes to languages or frameworks, we can quickly get to the edge of the Programming Language Wars . Any languages or frameworks selected are there to show examples of how something can be done. This does not mean that it is the preferred or best technology for every use case.
Frontend service We want to have routing, server side rendering, client side rendering, page layouts, error pages, easy to integrate components and much more.
Therefore, we decided to use SvelteKit which is based on Svelte . The implementation will be done in JavaScript.
If you are new to this field, I am pretty sure it will be interesting to get in touch with.
Reverse proxy service We have chosen nginx as our gatekeeper for all incoming requests. We configure a little bit with the configuration language for nginx .
We use nginx as a reverse proxy to forward incoming requests to the appropriate services. For private requests, we will first perform an automatic auth_request to avoid burdening our application with additional logic. The mod_rewrite module will help us distinguish between get and post methods within the same service.
We used nginx as reverse proxy to route incoming request to the appropriate services. For private requests we first will perform an automatic auth_request to not bother our application with extra logic. The mod_rewrite module will help us distinguish between get and post methods within the same service.
Database service A graph database paired with a JSON document store seems to be a good fit for our project. ArangoDB is our choice to achieve that. It has a user-friendly query language called AQL, a lot of built-in features and last but not least it is very well documented.
Session/Cache service For data with short lifetimes between microservices or requests, we use Redis as key:value store. However, we do not use the built-in session management here.
Authorization service For this tutorial we wanted to keep it small and simple with a widely used programming language and chose PHP and the slimframework .
We use additional frameworks for token handling (JWT), password generation, cookies, http requests, etc.
User service This will be the main service providing public and private APIs for CRUD operations on user-related data. It is also implemented with PHP.
Explore service Public available APIs like full text search or trending tags. Programmed with PHP.
Pre-parse service Since we allow Markdown in posts, we need to do some pre-parsing on the server side using Marked and jsdom for security reasons.
Express.js is used for our pre-parse backend service in Node.js.
Infos to participate along the series That's it for today. We've made some important decisions for our projects and outlined the microservice architecture we'll be building. Before we dive into the programming, we invite you to try these things out for yourself and share your experiences.
All source code and configurations will be explained throughout the series and are publicly available via Github.
We will also show how to run the project on your local system for development and later for production. The intention is to provide a step-by-step guide that you can follow. You can also contribute via Github and join our community on Discord . And last but not least, you can check out our working demo of the Twitter clone at twitter-demo.mogenius.com .
In the next article, we will set up the database and our Redis for session handling and caching.
Part 2: Building a microservice architecture from scratch to create a Twitter clone
