Welcome to Open Carnage

A resource for Halo Custom Edition and MCC modding, with unique means of rewarding content creation and support. Have a wander to see why we're worth the time! - EST. 2012

AntiMomentum

Member
  • Content count

    9
  • Joined

  • Last visited

  • Raffle Tickets

    0
  1. The Lua scripts go in: Documents\My Games\Halo CE\sapp\lua And in the sapp init at: Documents\My Games\Halo CE\sapp\init.txt You need at least two lines, the first line enables lua the second loads the script: lua 1 lua_load theluascriptname Note: for the init you don't put in theluascriptname.lua, so leave out .lua Then start sapp's haloceded.exe. Once the server boots somewhere near the top of the console it will say "Lua loaded" then "Sucessfully loaded theluascript!"
  2. This would be for when the gametype autobalance is enabled but you want to be on the same team with your friend in a public server (but I guess could be used for other stuff?) It would be great if there were a script that creates 8 pools, 2 max players each. Users enter something in chat to get put in a pool with their friend. Once some other player quits on either team it will automatically put one of the players in the pool to the team their friend is on (also in the same pool). Basically lets you automatically switch teams as soon as the team slot opens. This way if you try to switch teams to your friend but someone joins/autojoins in right after the player quits the script should work first. Just as an example: They'd enter just something like: join po1 If players are already in pool 1 sapp returns a message to the player that the pool is full. If a player joins a pool you are already in, sapp sends you a message to let you know.
  3. Thanks for reminding me! I've caught up on sleep lol. The highest spike I've seen is 2.4Gb/s (I will update the OP). Fortunately I know Linode (and probably DO) have higher incoming connection bandwidth than this! Even their lowest tier server have higher speeds, but 1 core alone won't drop the flood even with the bandwidth. And yep. He will just use floods as long as it works for him. He did not use any layer 7 attacks until I stopped the floods first, then he switched over to almost only only layer 7 attacks. This solution works for both And yep, sometimes he's just let the flood attacks run but usually stops it fairly quick after the server crashes. He might end up flooding for an hour once you're stopping him. Maybe his flood attacks are actually the newer ones for him since you've been seeing attacks before me. Interesting. Multiple protected servers with this method should wear out his flood resources pretty quick if he's letting them run that long though. It's much cheaper to have these high bandwidth servers than it is for him to carry out high band floods. Unless he built a botnet completely from scratch which I SERIOUSLY doubt. He hasn't even tried certain protocols I'd expect to see if he were targeting specifics. *edit* looks like he got me with a new flood he used for about an hour. Will recon and update next time he flood attacks. Traffic didn't go past 700Mb/s so it was probably a new protocol. Can't be certain until I'm capturing again, still no floods. Moved some source port drops to seven wall since they were layer 7 anyways to save resources and I wasn't doing that when successfully stopping past floods. New iptables are at the top of OP Edit** He's attacking at layer 7 rn lol. https://ibb.co/D70p34W https://ibb.co/bQB8p6k server survived the layer 7
  4. A concept: Windows 10 Pro and maybe 8 Windows Server ect can now run linux with Hyper V. You literally just enable some linux settings in Windows Programs and Features. With Docker, which also runs in Hyper V, it might actually be possible to do all of this IN WINDOWS with the settings enabled and Docker installed. Fairly certain the Linux stuff actually came in Windows update anyways. I would not at all be surprised if it requires more cpu to do this however. **11/2 update** New iptables kernel settings are available upon request. They'll be made public in a day or two if they still stand. You can message me on Discord: AntiMomentum#0262 I'll just copy and paste them into the chat (no links or downloads). The current settings on this post will no longer work completely. I'm going to break this down into numbered sections. After getting back into CE and making the server containers I thought my servers kept crashing because of Sapp in Wine in Docker or something lol. Nope. Devieth filled me in that someone has been ddos'ing this game I love since 2018. 1. So I'm going to post some iptables rules and kernel settings below that, with the hardware and bandwidth requirements, should stop the attacks from crashing your server(s). 2. I will explain one way you can watch the traffic and tell the general difference between the lower layer attacks and the layer 7 attacks (and normal traffic). 3. Then I will explain how to recon new DDOS attacks to your server so that you can adapt to future ddos attacks, and what server admins with servers that get hit often can collectively do. Not every last detail is going to explained. Some on purpose, because I believe this is a script kiddie doing this. And the other details because I'm too tired to get into putting in pictures to make this easier to follow for those who aren't that used to command line Linux. I'll get to that tomorrow or sometime real soon. After I beat his lower layer flood attacks I wasn't attacked at all for a day. Then he came back with the layer 7 attacks. He got me with them. Then I've beat those. It's now been another day so we'll see how he reacts soon. ** Little update here, sustained attacks at 800Mbit/s. Some new layer 7 attacks for udp games in general. 1. Requirements and Installation: Ok, at minimum you need at least dual core CPU (or two single core CPUs). On Linode I've been using the 2 CPU shared Linode server. In order to drop the lower layer flood attacks from touching wine/haloceded it is going to take about 120-140% CPU usage. Meaning 1 cpu/core and some of a 2nd as well. Fortunately, the actual layer 7 attacks themselves are pretty light on all the hardware as long as it gets properly dropped. As for bandwidth, I've seen some incoming brief spikes at around 1.4Gb/s. Even one at 2.4Gb/s! Sustained attacks at around 700Mb/s and less. You may seem a few Mbit/s of extra OUTgoing traffic from the layer 7 traffic and sometimes no extra traffic (not every is exactly the same). The CPU and bandwidth requirements are what's important. Linode AND DigitalOcean both offer $100 credit to new users. That's the environment I've been using (well just Linode, but Linode and DO are basically the same). The OS I use is Ubuntu 18.04 LTS. AWS and other providers have outside firewall rules and these iptables will almost guaranteed break ALL access to your server so you might need to manually change some of these outside rules if you're using AWS, Gcloud, and others. Personally, I haven't used those 3 yet but from research I've gathered some/all of their outside security/firewall rules must be applied before the OS you use first boots/gets created. Im using slashes there because I don't know exactly. First we need to install iptables netfiler. This will also install iptables persistent, but what I post is not saved after rebooting the machine. So if these iptables break too much functionality in your environment you can reboot to flush them out, or remove the end of sysctl.conf too if necessary (see below). I recommend NOT manually "COMMIT"ing these iptables because this is even going to break apt functionality. So we're going to install some not -required tools as well and to keep adapting to new DDOS attack scripts. You will need a way to monitor both bandwidth usage in real-time as well as a way to capture packets for recon if he tries new attacks. Let's dive in. apt install iptables-persistent netfilter-persistent modprobe ip_conntrack # Two optional but recommended tools these will be explained below: apt-get install nload apt-get install tshark I save the iptables rules to a text file and make them executable: touch ddosprotect.sh chmod +x ddosprotect.sh nano ddosprotect.sh Copy and paste the following iptables rules and save the text sh file. These rules use some docker chains. If you're not using Docker take out any line containing any docker names (like docker0, DOCKER-ISOLATION, DOCKER-USER ect) but leave the line blank rather than backspacing out the line. This way the number of lines stays the same for easy reference. This also assumes haloceded.exe is using default port 2302 but I will explain how to add/change ports as well (it's easy). For reference in this post for updates, line 1 starts at iptables -P INPUT DROP. The last rule is at line 68. We will not run these rules yet! The iptables are here: https://justpaste.it/7g7wj Next, we must change kernel settings to make these rules are completely effective. You can't stop the whole flood without them. Do: nano /etc/sysctl.conf and go down to the bottom of the file and copy and paste the below settings in, and save. ## These kernel settings also DO persist after reboot. However, they won't break things like apt and whatnot. You can remove them from the file and do sysctl -p to undo these settings: https://justpaste.it/8fkqv Now that you've copied those in and saved the file do this to apply them: sysctl -p At this point you can technically run the iptables script and start your halo server at the default 2302 port. But you'll want an terminal screen not being used by your halo server. screen -S empty screen -S newterminel wine haloceded.exe screen -S SomeScreenName<--- this just goes in front of whatever command you use to start haloceded.exe. To view all screens: screen -ls To switch screens screen -xr newterminel or screen -xr empty Pressing Ctrl a + d will detatch from the screen, and leave whatever command you used running Also works with docker run, and docker run -itd can be used as well without screen (thanks to Crash for pointing out -d to me!) I encourage you to finish reading this post first though unless you just want to make sure it actually runs, shows in the game server list, and you can join it in game. To apply the iptables rules: ./ddosprotect.sh At this point your iptables shouldn't show errors. You should have a terminal you can keep typing commands into. Your halo server still functions online hopefully lol. And now we're ready to dive into monitoring and adapting If your halo server freezes or takes a few minutes to load it's networking is broken and you might need to take out or alter some iptables rules. Assuming you've had it successfully working before trying these rules of course. Local and internal address drops are at lines 21-28. I recommend only taking out one at a time to see if it fixes things. I do want to point out that I didn't have the rule at line 11 before. I just recently added that since there was about a 5 second delay for the server to show up in the client list when Getting List/Refreshing. It fixes that 2. RECON Ok to watch incoming and outgoing bandwidth on the public network interface (probably named eth0 or enp0s3) do: nload eth0 there are multiple linux commands to check interface names if it's not eth0 or enp0s3 Watch nload and just see what the graphs look like for normal halo server network traffic when you have players in it. That's your baseline. It should be about like this with 1 full halo server: https://ibb.co/VxKt9tk Now here's what layer 7 attack looks like: https://ibb.co/zZ0tZgR but without the halo console output right there lol (the 2 players happen to quit there, services weren't interrupted). and another different layer 7 attack: https://ibb.co/KhPy8s0 Lower layer flood: https://ibb.co/CzQTzjQ I don't have a saved screenshot in nload of the flood attacks. But yeah they can easily be double that 300mb/s. Again can even spike into the 1.4 to 2.4Gb/s range. And can sustain at 100-800mb/s+ for 15, 20 minutes, and when I finally got the flood rules right he tried to keep flooding it for like an hour. Within about 10 seconds over 7 million packets are being dealt with at 600Mb/s+ Thankfully, only outgoing traffic goes towards the monthly network quota. For the 2 CPU server on Linode that's 2TB a month. Plenty. But there is more we can do collectively that'll save on costs and I'll get to that. But first let's see how to adapt. 3. Adaptation Today, tomorrow, next week, or never the attacker(s) may try different layer 7 attacks (or lower layer floods) that the current rules won't stop. That's ok! Wireshark (tshark), a packet capturing tool will tell us exactly which protocols we are being attacked with! There is one very significant catch. If you get lower layer flooded you can certainly capture the packets, however the CPU load will essentially double. Your halo server will crash if flooded while capturing at the same time without even more CPUs. Layer 7 attacks while capturing should be perfectly fine unless the current rules aren't working. But we will certainly get valuable info if rules aren't already stopping new attacks anyways so meh. So you can now tell difference between low layer and layer 7 and make your own decisions about that. So here's how we capture packets (assuming public interface name eth0): screen -S capture tshark -a filesize:10000 -b files:200 -i eth0 -w ddoscap1.pcapng This will capture packets and save them to files up until about 20GB of files are saved. You can alter the command to use more or less space. If you have more space it can help get info, especially if you intend to afk for a while. To stop the capture at any point do: screen -xr capture and then press Ctrl + c Then use filezilla or whatever choice you want to transfer the files over your PC with Wireshark. Wireshark will allow you to view the packets in a GUI to make things easy. Look at the protocols column. Now say you see a bunch of packets using the NDS protocol. If it was from a layer 7 attack we simply add a rule into the the SEVEN_WALL chain above line 46 like so: iptables -A SEVEN_WALL -m string --string "NDS" --algo kmp -j DROP Meaning, we can update this post here with findings of potential future ddos attack as we adapt with new strings. Also, to allow the server on a different port like 2308 replace line 43 (which is iptables -A INPUT -p udp --dport 2302 -j SEVEN_WALL) with iptables -A INPUT -p udp --dport 2308 -j SEVEN_WALL To allow multiple halo servers, make sure to add their rules at or directly under line 43 without removing anything else. They just get added in there. Now I have reason to believe that as far as the flood attacks go the person can only either do so many of those a month or something, or just gave up on trying to my server specifically. This means we probably don't need to use the 2CPU server the entire month. Linode and DO stop charging once the server is deleted. Meaning when they say the server costs $20 per month that assumes you are keeping it the full entire month. You can delete it at any time to stop. So we might be able to exhaust the floods. And even if he has some ddos account that renews every month, those admins with popular servers can do the 2 CPU for like a week and between us probably wear him out by then already. Because the layer 7 attacks on their own do not require much CPU at all hardly. At that point the cheapest of services will do.
  5. The stock dev cheat commands are infinite ammo and bottomless clip. You probably don't even need a Lua script for bottomless clip. I don't remember exactly but just enter something like: cheat_bottomless_clip 1
  6. No worries. Even plenty of admins I've talked too have only heard the name Docker lol. There are different uses, but being able to have the same environment and quickly is generally the idea. Like virtual machines. Especially useful for software developers, if they write code in one version of an OS they don't necessarily have to worry about it working in others. But in this case for example if you built images of your own halo servers and the game dns master server never goes down, your image will always work on a basic linux machine with internet (there are probably exceptions of course). You can even take the automation further with docker-compose. So say you have 6 different halo servers. You've built an image for each one. You'll never have to do that part again. From there on a fresh host linux install they can all be running in about 4 commands after you copy in the yml. apt update, apt install docker, apt-get install docker-compose, docker-compose up lol (My docker-compose file isn't working. After I deal with the root account issue I'll be fixing up the docker-compose.yml). The halo servers themselves don't have to worry about talking with each other. Docker handles the container networking. They are separate from each other as far as they know which can actually make things easier. Even in just network administration it's nice. Because once you have the containers' own network(s) set up that's generally done too even if you are using the same containers in different physical host machines with seperate networking info. Now imagine you're big data like Google. You now use Kubernetes to control your docker-compose/containers on a vast scale. Load balancing, containerizing databases, controlling hardware resources, ect. A developer pushes a code update, Kubernetes updates the containers safely and you move on. A user makes a new account and the account is accessible from anywhere quickly. Kubernetes takes extra configuration too though. I've never used it. But this is one way that sort of stuff happens
  7. So far the resource usage is light with 6 containers running. Granted, I'll know more at a time when players are on to fill at least one or two of the halo servers up. I'm less concerned with install size personally. And yes, to be perfectly clear, running as root is opening up to attack. Configuring it to not require root will be the next step for this project. At this point it is functional not secure. It's certainly not in a state where someone would want to set up a server and leave it. *Edit 10/2* I put an edit in op, but the antimomentum/haloce container now runs 10.2.1 with UPX removed.
  8. Here's the github: www.halopc.org **Edit 10/14/20** --cap-drop NET_RAW and --cap-drop NET_BIND_SERVICE can be used to give the container less-than-default sudo docker run privileges (see the link about the docker run command below) example: sudo docker run -it -p 2302:2302/udp --cap-drop NET_RAW --cap-drop NET_BIND_SERVICE antimomentum/haloce There are probably more privilege drops that work! Feel free to try them out and post your findngs! Updates will soon be labeled on a monthly basis to collapse this post a bit as well as moved to the bottom of this OP. ALSO, I recently built a massive container that has a mix of about 12GB of halomap's Top 50 maps for each year available. Of course, not every map in the top listS were used, I picked and chose from each year available. But there are also more recent maps added in, particularly big ones. And some others. Big/exploring maps is sort of the theme for this container. (Not going to lie the current mapcycle needs improvement) Don't forget to pull this or your own containers to make them available forever The container is again massive: docker run -it -p 2302:2302/udp antimomentum/top50 You will certainly need more than 25GB to actually run that container, not including host OS space. Use sv_maplist to see the maps available in the container. (The main container below is much smaller) **Edit Update 10/2/2020** The containers no longer require the --privileged full root access! Commands have been edited accordingly here and on the github Also, now runs sapp 10.2.x with or without UPX compression. The current antimomentum/haloce container now runs 10.2.1 without UPX Reduced docker file build size by removing unneeded layers, saving about 1GB in compressed download/build size. Here's some information about host access using the docker run command: https://docs.docker.com/engine/reference/run/ Running the docker daemon rootless: https://docs.docker.com/engine/security/rootless/ I'm making this post for what might be the easiest way to get a halo ce server up and running in linux. This method is 100% headless, and *might* even run in free tier cloud services. It's pretty light once it's running Full credit for the script, attached here to the post, goes to AugusDogus on github. I fixed the docker stuff. (start.txt is start.sh on the github page if you'd rather just view it). And full credit to þsϵυdø.þrø×϶n for removing UPX On the github page you can view "Dockerfile" to see the commands needed to get wine installed manually in Ubuntu 16 lts server (I did this in a host OS before doing it in docker, it works), at which point you can make the /game directory and use the script. halopc.com also has more information about manual installs. The script itself so far with a wine install has worked for me in both Ubuntu 18.04 LTS server and 16.04 LTS server. That's the manual way to do it. You can also check http://halopc.com/sv_extensions/ Here's the easiest way: apt update apt-get install docker.io Now you have two options. If you want to just get a halo server up and running asap: docker run -it -p 2302:2302/udp antimomentum/haloce Make some coffee Boom. Done. Assuming you don't have port 2302 blocked, or don't need to do your own usual port forwarding at home, ect. The halo server name will appear blank in the game's server list but it's a special character so the the server list arranges it to the top by Server Name. It's running on Bloodgulch. Don't forget to turn off your empty server search filter! Of course, you can change the server name using sv_name in the server console If you want to make your own server container images just check out the github. No need to rely on my container, you can make your own! I also recommend making a Docker account so you can save the container images you make to Docker! That way in literally THREE COMMANDS you can have your server container(s) up and running! (after you've already made them of course lol). I just went with a free account for Docker. If you want to modify anything including even the docker run command please post your findings for less privileges. The less privileges the better of course The docker run command can be given less than default access, please post your findings of least privilege! Whether it be the docker run command, script, or otherwise. To check your halo/sapp config itself, run sapp's command in the halo console: pl If it says it's a server function only! you messed something up, but not too bad to keep things from booting lol. Make sure you're using sapp 10.2 or 10.2.1 To run multiple container servers in Ubuntu 18.04 LTS server: For a VPS install Putty first on your Windows PC to connect to your server, then: apt-get install -y screen screen -S empty screen -S halo1 docker run -it -p 2302:2302/udp antimomentum/haloce press ctrl a + d screen -S halo2 docker run -it -p 2305:2305/udp antimomentum/haloce wineconsole haloceded.exe -port 2305 press ctrl a + d ect With your server images: screen -S empty screen -S halo1 docker run -it -p 2302:2302/udp YourDockerUsername/YourServerImage1 ctrl a + d screen -S halo2 docker run -it -p 2306:2306/udp YourDockerUsername/YourServerImage2 wineconsole haloceded.exe -port 2306 ctrl a + d screen -S halo3 docker run -it -p 2307:2307/udp YourDockerUsername/YourServerImage3 wineconsole haloceded.exe -port 2307 And so on Note: The halo1, halo2, halo3 names for screen are made up as you enter the command. No screen pre-configurations are done here Using Putty with the screen command allows you to close the putty window without your halo server containers closing *10/2/20*: Single core cpu usage runs at 20-40% with 2 full containers. Should be able to squeeze in a 3rd if you think it'll fill. If you think only 2 or less of your server containers will max out to 16 players, at the same time, you could run like 6+ containers on a single core machine with 1GB of RAM. start.txt
  9. https://github.com/antimomentum/haloce Scroll down to the step by step section. Also keep in mind that depending on the cloud provider you use, there could be security rules you need to set up outside the host OS to allow internet traffic to it. This is true for AWS, Google cloud services, and Azure if I'm not mistaken. Providers like Linode and DIgitalOcean don't have that extra step outside the OS. The github page should have all the info you need to run a halo custom edition dedi server with sapp and lua on Linode in Ubuntu 18.04 LTS server Here's the shortcut: apt update apt upgrade -y apt-get install docker.io docker run -it -p 2302:2302/udp -p 2303:2302/udp antimomentum/haloce Wait a few minutes, grab some coffee. Done. You are booted to the haloceded.exe console and the server is running with sapp, and will show up in the game's server list There's another section on that github page for installing your own server/sapp/ idk what else files to a docker build. My server using this container was full last night. CPU usage was at 10-21% and ram was under 300mb on bloodgulch. This might even run in free tier cloud services lol, maybe depending on what maps you use.