Introduction
MeshCentral is a free, open source, remote management and control web site built purely in NodeJS. Administrators can setup their own instances of the server in a few minutes and start working with hundreds of settings and customization options. MeshCentral includes everything from agents in Windows, Linux, MacOS, Android to built-in remote desktop, RDP, VNC, SSH, SFTP support and much more. Because MeshCentral instances are often exposed to the internet, security is of paramount importance. MeshCentral supports many security features like built-in Let’s Encrypt support, 2FA and more, but this does not prevent bots and attackers of various kind from trying to send all sorts of bad traffic to our instances, there is where CrowdSec comes in.
Experience with the public MeshCentral server
In the early days of MeshCentral, over 10 years ago now, I started by making the code public and launching a public server at https://meshcentral.com so that anyone can try MeshCentral without having to install their own instance. This public instance has been running ever since and served as a test vehicle for the code base, but also, served as an enlightening experience for what public facing servers can endure when exposed on the internet. When looking at the logs we could see a range of a few 100 IP addresses would, at regular intervals, try to guess usernames and passwords. Clearly, the attackers were using known leaked username and password pairs for these guesses. These were hitting the server from many addresses at a slow rate, every few minutes with guesses that probably existed on other services. At the time, I could look at the logs manually and see around 8 IP ranges that needed to be disallowed in order to block this attack which I did manually. However, this is not sustainable. A better solution had to be found.
Enter CrowdSec, the IP address reputation service
Since MeshCentral is open source and deployed by many people, I was not just looking for a solution that could be installed on my public MeshCentral instance. I was looking for something that would tackle this issue for everyone installing their own instances. One user on the MeshCentral GitHub suggested I take a look at CrowdSec and I was immediately interested. This is something that many MeshCentral administrators could deploy. One has to mention at this point that I suspect many users of MeshCentral are quite cost conscious and so, the CrowdSec community offering is going to be perfect for many MeshCentral users.
Implementing the bouncer into MeshCentral
MeshCentral is built entirely in pure NodeJS so that it’s easy to install and run on all NodeJS supported operating system and processor architectures. It uses the ExpressJS web server and has luck would have it, CrowdSec has a sample bouncer implementation on NPM that works with ExpressJS located at https://www.npmjs.com/package/@crowdsec/express-bouncer. I immediately got started on integrating the code into MeshCentral. MeshCentral instance administrators would need to configure a new section into the MeshCentral configuration like this:
{
"settings": {
"port": 443,
"crowdsec": {
"url": "http://localhost:8080",
"apiKey": "BOUNCER_API_KEY",
"fallbackRemediation": "captcha"
}
}
}
This new “crowdsec” section would be passed into the CrowdSec ExpressJS bouncer configuration as-is, so that not only would the URL and API Key be configured, but administrators could configure all of the other CrowdSec settings that are part of the CrowdSec express-bouncer package.
When it came time to integrate the express-bouncer into MeshCentral, I started by following the samples and added the bouncer to my ExpressJS routes. Something like this:
// Configure CrowdSec Middleware.
const crowdsecMiddleware = await expressCrowdsecBouncer({
url: "http://localhost:8080",
apiKey: "BOUNCER_API_KEY",
});
// Configure Express server.
const app = express();
app.use(bodyParser.urlencoded({ extended: true }));
app.use(crowdsecMiddleware);
This code will make it so that the bouncer gets to respond to any incoming HTTP queries before they are processed by the rest of the application. Because MeshCentral is complex and handles many types of queries, it’s not practical to assume that all HTTP POST requests coming to MeshCentral would be URL encoded, but the express-bouncer package made this assumption, so this line was required to make it work:
app.use(bodyParser.urlencoded({ extended: true }));
There were other issues too, MeshCentral handles web-socket connections and has its own code to determine the actual IP address of a request if MeshCentral is placed behind a reverse-proxy. I did not want to have the express-bouncer make its own separate attempt at figuring out the correct IP address. For this and other raisons, using the CrowdSec express-bouncer package as-is was not a good option.
Since all NodeJS packages are just source code bundles, it’s not difficult to look into the internals of now the CrowdSec express-bouncer was build and just use the parts you want to use. I ended up building my own “crowdsec.js” modules that references some of the internals of the express-bouncer package to make it do exactly what I wanted. You can see this code module here: https://github.com/Ylianst/MeshCentral/blob/master/crowdsec.js. It’s not a lot of code, and the upside is that I get exactly the behavior I want out of the bouncer.
A key part of what makes this work is that I could call the internal method getRemediationForIp() to ask CrowdSec what should I do with a given IP address:
var remediation = await getRemediationForIp(clientIp);
It returns “bypass”, “ban” or “captcha” and I could then handle these cases exactly like I want to. For example, when “captcha” was indicated, I could redirect the user to a “captcha” URL on MeshCentral where it would then be handled correctly with the response POST body being URL decoded.
Testing the MeshCentral bouncer implementation
The CrowdSec CLI tool “cscli” has all of the features I needed to quickly test my new bouncer. While most MeshCentral instances run on Linux, my own development environment uses Visual Studio on Windows and so, having the cscli tool installable on Windows was wonderful. All you did to get the API key is:
cscli bouncers add
This returns the API key from the local CrowdSec web server. I could then add the API key into my MeshCentral config.json file, restart the MeshCentral server and I was up and running with CrowdSec protection. To test it, the CrowdSec CLI provides a way to manually add decisions, so all I had to do is use some of these commands:
cscli decisions add --ip 1.2.3.4 --type captcha
cscli decisions add --ip 1.2.3.4 --type ban
cscli decisions list
cscli decisions remove –ip 1.2.3.4
I would then refresh the login page on MeshCentral to see various reactions.
After a few bug fixes, the results look great and I can’t wait to have this deployed in production. This will add a new layer of security to my own MeshCentral instances and the instances of many of the MeshCentral users and community members.
Future work
There are some improvements I will surely work on in the future. One thing I noticed when running the decision command using the CrowdSec CLI is that the decision scope does not have to be an IP address, it can also be a username. So, you could type this:
cscli decisions add --scope username --value baduser –type ban
This would allow CrowdSec to make a decision on a user name. I don’t know if this is widely used and the CrowdSec express-bouncer package does not have an equivalent call for usernames as it has for IP addresses with getRemediationForIp(). However, I could implement this in the future if there is community request for it. You could then block or CAPTCHA a specific username.
Conclusion
Implementing the CrowdSec bouncer in MeshCentral took less than a day and it was well worth the effort. MeshCentral is licensed under Apache 2.0 and anyone is welcome to take by own code as starting point for their own project. CrowdSec provides an added layer of security that MeshCentral users can easily take advantage of. Because of my previous experiences with the MeshCentral public server, I expect the MeshCentral bouncer to get some use going forward and that is one less thing I need to worry about.
Opinions expressed are solely my own and do not express the views or opinions of my employer.
About the author
Ylian Saint-Hilaire in Principal Engineer at Intel, Ylian Saint-Hilaire is part of the Business Client Platform group working on web based remote computer management and peer-to-peer networking.
Ylian is known for many projects including the Meshcentral.com website, the Manageability Developer Tool Kit (MDTK) and Intel Tools for UPnP Technologies. Recognized as an innovator and public speaker, he was awarded two Intel Achievement Awards for outstanding work enabling the digital home.
Specialties: Software Design, Software Engineering, Public Speaking
