Using Run Length Encoding (RLE) to Compress ROBLOX Terrain

By -

Terrain-based ROBLOX IslandROBLOX released its high-performance terrain feature late in 2011, allowing users to create and play games significantly larger than anything previously seen on the platform. Many ROBLOX users have since taken advantage of terrain, particularly to facilitate expansive games with fast vehicles. Currently, the ROBLOX Content Team is pushing the feature further than ever before with a new game that procedurally generates destructible, cliff- and gold-ridden terrain each round (we’ll have more on that soon).

Terrain can be up to 2048 x 256 x 2048 studs in dimension, containing up to 16 million 4 x 4 x 4 terrain cells, with no graphics slow-down. One of the supporting techniques we use to make terrain so scalable and efficient is Run Length Encoding. Run Length Encoding makes the file sizes of terrain places a small fraction of what it would be with normal parts.

Run Length Encoding

Every published ROBLOX game is hosted on Roblox.com, so one of the keys to efficient terrain on our platform is saving the data in a cost-effective way. To that end, we use Run Length Encoding, or RLE. RLE is a method of storing a sequence of duplicates. Rather than store each duplicate item individually, RLE creates a count of the duplicates.

Let’s say we’re saving a small piece of terrain that consists of eight grass cells. You could store each grass cell individually – in this case, you would use five letters for each cell, for a total of 40 letters.

Grass Grass Grass Grass Grass Grass Grass Grass

Using RLE, we can store the small piece of terrain much more efficiently. Using only seven letters, we reduce the overhead by more than five times.

8xgrass

RLE doesn’t always save space because it’s dependent on there being a sequence of duplicates. For example, suppose you have alternating solid and empty spaces in a voxel grid. Again, you could store each cell individually.

solid empty solid empty solid empty

But using RLE would actually increase the data you’re storing by 12 characters, or about 40%.

1xsolid 1xempty 1xsolid 1xempty 1xsolid 1xempty

Duplicate Terrain Cells (Bottom view)

ROBLOX terrain is usually composed of a sequence of duplicates. As you can see in this screenshot of the bottom side of a common terrain formation, there are a lot of repeated cells.

A Practical Example

For a more practical example, consider two 15 x 15 x 15 structures, one made of standard ROBLOX parts and one built with terrain cells. Here’s a comparison of the amount of data stored with and without compression.

Regular ROBLOX parts Regular ROBLOX parts (Zipped) Terrain Terrain with RLE Terrain with RLE (Zipped)
8245 KB 83 KB 32768 KB 44 KB 3 KB

The unencoded, uncompressed terrain structure is actually more data intensive than the structure made with parts, but RLE and compression make the terrain structure much more efficient. Terrain with RLE and compression is 27 times smaller than compressed regular ROBLOX parts, 14 times smaller than terrain with RLE but no compression, and about 11,000 times smaller than standard terrain. ROBLOX uses zipped RLE terrain in practice.

Using these ratios, if you built a maximum-size terrain out of regular ROBLOX parts, the game file, even with compression, would already be around 380 MB.

We should note that much of the data for terrain is implicit; each cell’s position is implied by its terrain coordinate. That means we don’t need to save its position in the file. Parts, on the other hand, are physically simulated, so their starting position needs to be recorded and updated during the simulation.

Alternatives to RLE

There are many alternate compression techniques. Variations of RLE are commonly used for applications where speed to compress and speed to decompress are more important than average compression ratio. For example, Snappy, Google’s compression/decompression library, does not aim for maximum compression, but very high speeds and reasonable compression. At its core, Snappy uses RLE, with a few other techniques layered on top to further improve compression and decompression.

But standard RLE is a good fit for ROBLOX, and one of the key reasons why terrain, despite its expanse, doesn’t slow down the user experience.

222 thoughts on “Using Run Length Encoding (RLE) to Compress ROBLOX Terrain”

    • ahem?
      do you think they are copying minecraft?
      is there slopes in minecraft? no.
      is there water. yes but doesnt react with other blocks. ROBLOX is not copying Minecraft.

      • If roblox was trying to be like minecraft there would be minecraft type enemies pickaxes to mine things and other stuff roblox is just trying a realistic terrain if you guys don’t like terrain then you don’t have to use it other people like me like terrain.

      • Are you hearing yourself? ROBLOX has implemented different features just so they can get less of ‘it looks like minecraft’. When in fact Terrain generation came out late minecraft beta where the game was beginning to become very popular. Since its’ release, ROBLOX has had essential ‘mine/dig’ and ‘place block’ tools, as well as some elements similar to WorldEdit, a mod that is designed for mass Terrain modification. Sure, minecraft has stone and ores, which roblox is attempting to implement, and minecraft has squares not slopes, and water reacts differently, but the fundements, the random generation codes, and Terrain modification is almost exactly the same. And mobs are no loophole either. A player could easily script the basic AI for a minecraft pig or creeper, and remake its’ mesh as well. The focus is on TERRAON GENERATION.

      • Agreed. It’s just to help your place look better than just a silly old baseplate with buildings.

      • I argee with the above.Roblox is updating once a while.They are trying to make MORE realistic.Minecraft is just blocks.Know your diffence.

  4. Well, even if ROBLOX used RLE on normal blocks, it would still get laggy after a while because every block has a position, which takes up a lot of space. The reason ROBLOX uses RLE on high scalibility blocks is because they want to promote them, and honestly, the terrain blocks are better. The only problem is that with normal blocks you can make a brick look like almost anything. High scalibility blocks only have a few materials, but you can still make epic games with them. I have your back, ROBLOX!

    • Fail, in the last line I meant to say, “I’ve got your back ROBLOX!” xD

    • I prefer NORMAL blocks, it so mush easier to make anything.. and terrain keeps doing the wrong thing.

      so both are good in their own way..

  5. You guys realize that water has only just cone out, right? I’m sure they’re improving it whenever the Manatees grow tired of formulating hat ideas with rubber balls with random words written on them.

    • im hoping that they use the same things on ewater as the other high scability blocks because whenever i go up hill it just looks…. blocky not like flowing water. i dont want rapids in my water unless people can make boats there xD

  6. Um, can you guys return the old Studio Solo Mode system and NOT the Play ROBLOX layout?

  8. Way to keep stuff neat and easy.

    I’m sure a lot of people are gonna appreciate this.

  10. I don’t like stamper tool! i want the insert tool again! all want the insert tool again!

  11. Here is an idea. I Understand that terrain is extremely laggy witout RLE, seeing that its nothing more than textured bricks. How about applying RLE to regular Bricks? If I can run terrain with realitivly low lag compared to making stuff with bricks, why not apply it to bricks as well? Roblox would be even less laggy if RLE is applied to more than just terrain.

  12. 1 Problem with ROBLOX.

    The stamper tool is giving TOO MANY misconceptions to new ROBLOX users, why not use some of the old tools? Those were way better!

    • thats what the 8 tool is for and the givebtools/all is for my strange person in which i do not know.

  13. so, basically, you’ve made it so the least used thing is the fastest loading thing? good job. Wouldn’t it have been more efficient to find a way to use RLE for the really really big files, the Roblox bricks?

  14. Terrain, terrain, terrain. Seriously, ROBLOX, are you trying to copy MINECRAFT? Not focusing on other things?

    • Razor 1911′s demo, “We have accidentally borrowed your votedisk” generates terrain during a part of the demo.
      Is that ripping off Minecraft in your eyes? It is to you.

    • Minecraft and many other games got the idea of terrain from real life terrain (nature). Many games have terrain. Is it bad to have terrain?

      • over my dead body. if its bad to have terrain then i say we cant exist. terrain includes TREES. without trees we cant live.

  15. > Comparing space requirements of uncompressed XML format parts with tons of useless properties against RLE compressed array of bytes.

  16. Now admins have to change the name as well to RoRealisticPhysicsGame (RRPG). It’s supposed to have blocks, not reallistic terrains and life-like physics and stuff! Why do you do that?! You just lose old good players and make new spoiled and stupid players enjoy it. Sucks, doesn’t it?

    • I PARTLY agree. The name Roblox is a junction of “robots” and “blocks.” These “realistic terrains and life-like physics” are optional. You don’t HAVE to build with terrain. Besides those ARE blocks. Just a different TYPE of blocks.

    • Terrain and realistic physics does not destroy the purpose of Roblox. If a brick falls with realistic physics, what’s the problem with that? Is there something bad about realistic physics in Roblox? Nothing! They still use bricks. Lego is about bricks. If a drop a Lego brick and then falls with real life physics (because it’s real life), does that mean Lego is not about bricks because it falls down with real life physics? The game has water, guns and other stuff. Does that mean it’s not about bricks anymore because it has those? Terrain grass is also special BRICK. Cut your irrational argument and enjoy the game.

  17. I wish admins never changed ROBLOX and let us play the ROBLOX we played in 2008! But unfortunately, every good thing has an end.

  18. Ya thats great! When I make new places with terrain it is alot faster. One problem though. I CANT GET IN MY OLD PERSONNAL SERVERS! Could you maybe put some of the roblox studio tools in personnal servers so we dont have to use giant bricks all the time?

    • yea your right cause i have like 10 personal servers in nonuse right now and i have a few ideas and i need to finish those place

  19. this is great! I love it, just one thing, can you fix the exploit that ruins games?

    • They’re working on Studio 2.0, you know that right? Current Studio is five years old. And you know how hard it most likely is to make that stuff? Hard.

  22. Haha Minecraft is twice as better than roblox that is why i quit roblox and play minecraft now but i still check in once in a while and Roblox stop writing all these things i thought roblox is for kids..not teens so what kind of kid cares about that roblox…THINK?!

    • All you can say is that roblox is updating its terrain and show the baseplate data and all that and there we go..

  23. “…Games can take up 380 MB…” In Minecraft they already take up that much and the game is A TON more laggy than ROBLOX.

    • If they did. It would sadden to see ruin games. I will quit Roblox if they did removed Bricks.

  26. What’s so special about this? I would much rather have a scripting update rather then a blog about RLE compression. Sure, it was interesting, but where are our new features that we can completely use and customize? The water was nice, but it’s Terrain. It takes away scripters ability to fully manipulate it. You can still do amazing things, but I’d think it would be awesome if water was an object, a derived class of BasePart or something. If you read it, this isn’t anything new. It’s actually just an explanation. The in-game character changer can be delayed, the trading system can be delayed. I want features that you can script with. Possibly more dynamic explosions, or updates to humanoids. I don’t want explanations of a bad feature, I want new features.

  27. cool we can have 14 million terrain parts in our game

    too bad terrain isn’t useful at all

    make it so we can have 14 million PARTS in our game, because you all lied at the ROBLOX Rally by saying “we can have 14 million parts in our game”

  29. bug with water
    well when i put water in the game and i wanted a hill so when i used the orb tool to make a hill the water went up with the land

  33. I loved the terrain feature so much that since it came out i made a free map a month for all robloxians to take. Up until todays update june 14 2012 they worked flawlessly , but now I can’t load any of them. I can start the game, it goes to the fullscreen ,does the part count and freezes. They all work fine in test mode just not on server.

  34. ROBLOX, for the update that has just been caused, YOU BROKE THE TOOLBOX AND INSERTING PLAYER CREATED MODELS AND PLACES. You guys are just making the classic users LEAVE. You guys got ROBLOX popular at the start. NOW people are LEAVING because of ALL of THIS. I want you to fix the issue please, or many classic users would leave.

    Older accounts I had: BatMan, Tegwanee, 129675, and 456342.

  37. I haven’t seen any high quality games that use terrain. Crazyman32′s games are famous for their BRICK planes, not terrain.

  39. I may be smart but I don’t understand anuthing other than a new game is coming.

  42. I can’t wait until Roblox releases infinite-size terrain! 2048″256″2048 is still too small!

  44. Wait, my comp screen is 10 feet away so I cant read. you trying to make it unlimited?

  47. This is great and all- and I’d love to try it out- but I have Windows 7, so that means I can’t use Studio. :/ PLEASE fix this glitch ROBLOX.

    • I use windows 7 and I have no problem with the studio. It may be your computeris to old. Remember you have to go throught the start window to get to the studio. Its: Start-All Programs-Roblox-Roblox Studio. Hope that helps.

  48. COOL.
    but it kind of sounds like MINECRAFT. as i have said a MILLION times before, give the PLAYERS some power and let us have more OPTIONS!

  51. 1 Problem with ROBLOX.

    The stamper tool is giving TOO MANY misconceptions to new ROBLOX users, causing them to leave.

    Yeah, you should bring back the old tools!

    ADD THE STAMPER ITEMS TO THE ROBLOX SETS ON THE INSERT TOOL!

  52. Sooo, I somehow understood all of that… But in the end, I’m thinking: Wait, what? TERRAIN WOULD BE THAT LAGGY WITHOUT THIS? I hope Minecraft has used RLE, otherwise this makes ROBLOX less laggy than Minecraft. I’m serious.

  53. interesting how you do this roblox i understand it sort of but this is really cool

  54. Next task for Roblox devs IMO: Work on the actual main part of the game to reduce lag when moving thousands of parts so even Pentium 4s have almost no trouble.

    • >so even Pentium 4s have almost no trouble.

      Get out.
      Now.
      You don’t belong here.

  55. I have a feeling ROBLOX is trying to copy minecraft >_>

    • What do u mean ROBLOX is trying to copy Minecraft? Minecraft is boring,u know how? U have to stay on this place with only a few colors espeialy green.But on ROBLOX,there are millions of different colors.On minecraft u can only mine stuff and build.WHO CARES IF THE STUFF ON ROBLOX IS HANDED TO U!!!!! it doesn’t MATTER!!!!! It doesn’t make a difference.U Minecraft lovers just say that so u have something to say about ROBLOX (Please don’t ban me) On ROBLOX,u can Roleplay family,be a baby,work at places,build,drive,paint,create,etc. Not like minecraft,u only to 2 lame things. In Minecraft,u have to pay $20to have multi-player mode,but in ROBLOX,u can have a account and multi-player for FREE.ROBLOX graphics are WAY better so u know what the thing is.In Minecraft,I can’t tell what a sheep is on there. I know people say graphics don’t matter but it really does.If u don’t know what a thing is,it effects gameplay and gameplay matters ALOT. On ROBLOX, u can make your clothes(If u have any type of BC) and Minecraft,u can’t even customisze your character! Now u know why ROBLOX is WAY better then Minecraft,I’ve played Minecraft before and it’s just lame! And ROBLOX IS NOT TRYING TO COPY MINECRAFT!!!!

  56. If so, then why is it that a hollow Terrain building has less load penalty then a solid one? Water needs wedges and other shapes, and Terrain also needs upside-down wedges/cornerwedges.

  57. how does the network terrain update its voxels? using RLE to transfer data updated through a somewhat large explosion seems inefficient.

  58. “Terrain can be up to 2048 x 256 x 2048 studs in dimension, containing up to 16 million 4 x 4 x 4 terrain cells, with no graphics slow-down.”

    I personally beg to differ. I can’t use terrain without my computer freezing. Other than that, nice work.

    • You’re talking about generating terrain? I have the same problem but playing in terrain is awesome.

  59. They should really get sorcus to completely finish the gear Romaze though,

  61. I still lag from the terrain. Mostly when there are too many of those blocks.

  67. I don’t think terrain should have any limit to size.

    I mean yes, prehaps a massive limit like 40kX40kX40k to save memory but not anywhere near 2kX400X2k.

    Roblox uses that cloud thing anyway- nothing to worry about on the memory side.

  69. Is it possible to make the terrain movable off its Grid because I cannot get it to align with my place which used blocks.
    Also, is there the possibility of making terrain re sizable instead of just the huge square blocks that can be placed where-ever. It just makes integrating Terrain into old places extremely difficult. This extends to water – why not just make a BRICK?

  70. Wow! Very Cool I never knew how much data it took but now with the RLE I can make even bigger levels!