Jump to content
NQ-Nyzaltar

DevBlog: LUA Scripting and Distributed Processing Units (DPUs)

Recommended Posts

(Posted Friday 18th of September 2015 on the DevBlog)
 
devblog-dpu-lua-script-small.jpg
 
Big news! We just finished designing and implementing one of the most critical and distinguishing feature of Dual: the capability to script any construct that you make. A construct is anything that you build in the game, it can be a ship, a building, a space station, or whatever. It is a physical object, it can move, collide with stuff and potentially include hundreds of "Elements", which are operational units that you can craft or buy, and freely position inside your construct to add functionalities to it. Examples of Elements are: propulsion engines, control units (computers), doors, weapons, batteries, containers, accelerometers, radars, targeters, drone bay, elevator, and many more. The way the construct orchestrates all these Elements is through scripts, and what we call "Distributed Processing Units". Let's dive into it, and see what it can do.
 
Warning: The following Blog Post implies you already have some basic programming knowledge.
If you don't, some parts may seem a bit obscure or difficult to understand.
 
This is going to be a slightly more technical and longer post than usual, but I hope people interested in the topic will find it entertaining. In any case, remember an important point: you don't need to understand anything about scripting or ship building to actually buy a ship and fly it. You actually don't even need to use a ship at all if you don't want, there are plenty of activities to do besides this in the game. It's a bit like in real life: you don't need to be a mechanics to drive a car, and some people just never drive anyway, they take a taxi. It all depends on your playing style, and what you expect from the game. But if you are interested in creating your constructs and scripting their behavior, I hope you won't be disappointed by the deepness of the gameplay experience we propose.
 
So, let's summarize again what a "construct" is: in Dual, you can assemble chunks of matter (stone, wood, iron, kevlar, etc...) in any way you like, a bit like in Minecraft, but instead of simple cubes, you can forge almost any shape you like. The technology behind this is called "Dual Contouring". This can be used to build the hull of a spaceship, the walls of a castle or a gigantic statue. Up to you. Attached to this inert structural skeleton, you can add some components called Elements, which are actual gameplay components that you can deploy in your construct to make it functional. In a spaceship, you need propulsion engines, fuel tanks, navigation instruments, one or several cockpits, etc. All of these are Elements. The collection of all Shapes and Elements are what constitutes a construct, which is a whole that can be moved once created and is subject to the laws of physics (for example, it falls if dropped in a gravitational field). A construct can be owned, traded, copied, etc, but this is another story we will discuss in another post. Right now we will concentrate on one particular aspect: how do you orchestrate all the Elements so that they can work together? We will take the particular viewpoint of a spaceship, which is a good way to illustrate the concepts involved, but this can apply to anything you like, including a giant spaghetti monster robot or whatnot.
 
In terms of game design, we could opt for an easy strategy here. If you have the required number of engines in the right direction (no matter where they are), and you check the list of instruments needed, it would "magically" fly. With this approach, all ships would fly the same. Trying to put more engines, or optimizing their position would be more or less useless. Hoping to have an AI helping with automatic navigation would be up to the engineers of Novaquark only. Fancy a new way to drive your ship? Impossible. How about the weapons system? How about drones? All this would be predefined and more or less rigidly identical for all players. That's not what we have in mind for Dual Universe. While we will provide basic templates to start with, you?ll be able to engineer your construct the way you want. Engines are real (they physically push your ship where they are, with the power they have), gravity is real, weapons have to turn and target (which also requires a targeter). If you are smarter than others, you can get the job done in a better way, get an edge in battle, or in trade by launching the new Falcon X-42 superfighter and change the balance of game combat with new tactics and possibilities. It?s not only about how you can use the predefined capabilities of ships within a predefined classical game setting, but it?s also about how you can redefine these capabilities. We call it: emergent gameplay.
 
Each Element is an active unit. It can do basically two things: emit "events" and execute "functions". Let's give some examples: a radar unit can emit events like "new enemy detected at (x,y,z)", a jet engine can execute a function like "set thrusting power to 45%", a weapon can execute the function "fire", etc. Technically events and functions are described a bit the same way: a name followed by parameters written between parentheses. The above examples would become something like:
 
enemyAt(x,y,z)
setPower(45)
fire()
 

Events are emitted spontaneously by the Element when something happens for which it has been designed to react to, and functions are services that the element can fulfill when it is asked to by some external agent. When considering a particular Element, you want to know the list of events it can emit and the list of functions it can fulfill. This is entirely defining what this Element is, from the point of view of the gameplay. We call it its "Type".

 
Now, how do you orchestrate the interactions between several Elements, each emitting events and reacting to them, fulfilling functions in return, etc? The central notion we introduce here is what we call a "Distributed Processing Unit", or DPU in short. A DPU is a bit like a computer program, an orchestrator. It has several slots in which it is possible to plug Elements, and it contains a list of event handlers that can react to events emitted by the Elements plugged inside its slots. The schema below illustrate this:
 
devblog-dpu-001.jpg
 
Events handlers are conceptually quite simple: they are "condition => action" managers. On one side, there is a conditional filter, which is a generalization of an event from a given slot, with certain parameters set to given expected values, while other parameters can take a variable "free" value. A particular event emitted by an Element in a slot will be examined by all event handlers, and it will trigger the event handlers if the event signature matches the event handler filter.
 
To give an example: suppose that we have a filter like ?enemyAt(x, 42, y)?, associated to the ?radar? slot. Now the radar slot emits the event ?enemyAt(11,42,66)?. This event matched the filter and so will trigger the event handler. If the event ?enemyAt(12,13,66)? is emitted however, it will not match the filter (because 13 is not equal to 42). The schema below illustrate this point:
 
devblog-dpu-002.jpg
 
On the other side of an event handler is the action that the handler can trigger when the filter is matched. This is where LUA really enters the scene. The action is simply a piece of LUA code. LUA is a very simple and efficient scripting language, for which you can get many tutorials online. You can try this for a start.
 
Now what kind of code will you write on the LUA side? Basically anything you want, but, as you guessed, it will ultimately contain some calls to functions among those provided by the Elements plugged in the DPU slots. The syntax will be something like: "self.engine2.setPower(42)". The "self" prefix is a requirement of LUA, then comes the name of the slot, and then the name of a function available from the Element in that slot, together with its parameters between parenthesis.
 
That's it. That's the basics of what a DPU is and how scripting works. Let me summarize: a DPU is an orchestrator, it has several slots in which you can plug Elements. You can then define event handlers to catch events emitted by these plugged Elements, and react by executing LUA code, which includes calls to functions taken amongst the set of available functions of your plugged Elements.
 
To be more precise, I have to refine this picture a little bit: I talked about customizing a DPU, but exactly what DPU are we talking about? Where is it? In fact, the DPU you want to customize is stored inside a special Element called a "Control Unit". The DPU is started when you activate the Control Unit (go next to it and press the activation key). Notice that there is no problem with having several Control Units (hence, DPUs) inside the same construct, potentially all activated at the same time.
 
Now, the DPU inside a Control Unit is special because you'll have a GUI to freely customize it (plug stuff into the slots, define event handlers, etc). Actually, you can do even more than what I just described: you can define a set of events that your DPU is capable of emitting (event emission is done via a special LUA syntax inside your scripts). You can also define a set of functions associated to your DPU, and the corresponding LUA code that should be executed when this function is requested. If you remember, I presented the type of an Element as the set of events and functions exposed by this Element when plugged inside a DPU. Actually, the truth is that there is in fact a DPU inside each Element (albeit not a customizable one), and the events and functions of this Element are actually the events and functions of its DPU. Now to the important conclusion: it is not really Elements that you plug inside the slots of your Control Unit's customizable DPU. It is in fact DPUs. It can be the DPU of Elements, when you plug Elements, but it can be more generic or abstract DPUs. More about this now.
 
One particularly important "abstract" DPU is the System DPU. This DPU is capable of emitting events when keys are pressed. You almost always want to plug the System DPU, because you want to control your scripts with keystrokes bound to actions. So, typically, you will have several event handlers in your custom DPU to catch action events emitted by the System DPU. The System DPU is also capable of emitting "timeout" events when a timer is due, or at regular intervals, and many other useful functionalities.
 
You know enough now to script a simple spaceship controller. You need to put a least one Control Unit in the ship, and customize its DPU. Inside this DPU, you will plug the System DPU (typically in the "system" slot), plus at least 3 engines pointing upwards to lift the ship, and 2 more engines at the back of the ship to make it move forward. You need also to plug a gyro (more exactly: an inclinometer), which is an Element that provides the getPitch and getRoll functions, to know about the pitch/roll angles of your ship. And that's it. Now you set a timer with the system DPU (call the system.setTimer(0.1) function to set the update every 0.1 second) and catch the corresponding timeout event. When caught, you can then associate the LUA script that will query the gyro about the pitch/roll and use these values to correct the intensity of thrust power in your motors (using the setPower function) to stabilize the ship tilting. And of course, you will also catch the action events from the System DPU to inform your script about the direction in which the player wants to stir the ship. The exact details of how this is done in terms of control and dynamics are beyond the range of this post, but it involves simple Newtonian physics about torques and forces.
 
This above custom made Control Unit DPU could be seen as a black box, a ?Stabilizer?, with 7 slots (5 engines, 1 gyro and the System DPU). You can move this black box in what we call a Component DPU, so that you can sell/exchange it. Now players possessing your ?Stabilizer? can plug it in their own Control Unit DPU along with their own Elements, and then connect these Elements in the Component slots, with a simple drag and drop interface, or a smart ?autoconfigure? system, making the whole process of scripting a ship very simple for those who don?t want to put their hand on code:
 
devblog-dpu-003.jpg
 
The DPU system goes ways beyond this simple ship stabilization example. Weapon control, whether in FPS direct view on a jet fighter, or in tactical overall view in a battleship, will be handled with DPUs. Scripting a drone can be done with DPUs. Setting the automatic defense mechanism of your castle can be made with DPUs. Factories can be automated with DPUs, etc. It's all about orchestrating Elements and Component DPUs via scripts that react to events and execute LUA code. The user interface is also scriptable to decide what happens when the Control Unit is activated, how Elements can display their status and give access to parameters (we will probably talk about this in another blog post).
 
Novaquark will be providing several useful "starter" Component DPUs to start with, as well as smart autoconfigure options to handle the most basic cases. You can get a ship to fly without knowing a thing about DPUs. But, if you are interested in this aspect of the game, it will be up to you to build from there and create the most amazing control system and contraptions, win the markets or the wars with them and leave your mark in the Dual Universe history of innovation and engineering!
 
JC Baillie,
Project Lead.

Share this post


Link to post
Share on other sites

This was actually very good :D Even though my knowledge in Programming is very basic, I understood this pretty well :) The entire Scripting system that Dual Universe will have now sounds very interesting and awesome because, as a builder, I can make my creations really come to life :D I really liked how one person can make a DPU and trade/sell it to other players. This will open up alot of possibilities for players that are really good at scripting to create their own DPU business in the game :) Overall, this was very interesting and now has me really looking forward even more to the Scripting aspect of the game :D Keep up the great work! :)

Share this post


Link to post
Share on other sites

Man this is amazing. I don't know jack about scripting or lua or anything (I am a visual GUI kinda person) but I know some people who do. I hope I can get them into the game :D.

I have spent countless hours in another building game, sometimes months on a single project, but at the end of the day those builds do nothing but sit there and look cool (or not). The ability to make somthing that works in DU is stupidly cool. Outside of spaceships, mechs, land vehicles, we can make automated bases, defense towers, etc. I can't wait to play this game when it comes out!?

Share this post


Link to post
Share on other sites

I just read the Dev Blog again on LUA Scripting. I can't wait to play DU. I have made a couple hand full of spaceships in another voxel building game, but they only fly in my imagination because there is no development going on to make builds do anything more than look pretty. 

I can't wait to get into DU and make my first tiny attempt at a spaceship. Will most likely be something small, easy to make and get up and running. If/when you guys pull this off, DU is going to be the coolest game in existence!

Share this post


Link to post
Share on other sites

I just read the Dev Blog again on LUA Scripting. I can't wait to play DU. I have made a couple hand full of spaceships in another voxel building game, but they only fly in my imagination because there is no development going on to make builds do anything more than look pretty. 

 

I can't wait to get into DU and make my first tiny attempt at a spaceship. Will most likely be something small, easy to make and get up and running. If/when you guys pull this off, DU is going to be the coolest game in existence!

Every single time I read that post, it just makes me fall on the ground having a nerdgasm.

It always leaves me feeling good for a few days XD.

Share this post


Link to post
Share on other sites

I really love the concept, it seems so sophisticated and well put together.

Still I have some questions:

 

 - First of all, will I be able to let my programs store data gathered on "data cards" or something like this? In my opinion it was awesome if you could have a scanner on your ship which you can program to make a star map or gather blueprints of other ships (Of course you'd need a DPU to gather data and to store it in a certain way, and one to interpret it and maybe show it on a GUI)

 

 - The second question I have is about the physics: As they mostly involve both mass of the spaceships and mass of the planets (I'd guess the planets have different sizes and gravitational constants/masses) it'd be very nice to have a way of measuring it (except manually by measuring how long it takes to fall to the ground)

Share this post


Link to post
Share on other sites

I hope this isn't too broad. I mean, ship building and flying could be a game all by itself! If this is one of the pillars of the game it makes sense, but lets not forget about the rest of the "stuff to do" :D

Share this post


Link to post
Share on other sites

So just discovered this game and it's basically close to my dream game, in part because of this feature.

 

I have a question that i believe has a big importance on the script feature :

How are we limited in computing power ?

 

You arent the only one man, and welcome. 

 

In short, we dont know. The devs have stated they will make the determination on limits later on down the line if needed. 

 

Personally I hope to be able to write some complex and lengthy scripts. Although I dont expect them to take up that much CPU, just be on the complex side.

The idea seems to be just to write the interface between 2 components, one provides some initial data and you determine how the other acts based on that dataset. 

 

Im sure if people start writing scripts that just run CPU cycles to cause lag, something will be implemented to limit it. 

Share this post


Link to post
Share on other sites

You arent the only one man, and welcome. 

 

In short, we dont know. The devs have stated they will make the determination on limits later on down the line if needed. 

 

Personally I hope to be able to write some complex and lengthy scripts. Although I dont expect them to take up that much CPU, just be on the complex side.

The idea seems to be just to write the interface between 2 components, one provides some initial data and you determine how the other acts based on that dataset. 

 

Im sure if people start writing scripts that just run CPU cycles to cause lag, something will be implemented to limit it. 

that would be nice. That would allow us to write some thing close to a swarm A.I for drone or some bad ass predictive algo for a targeter xD (Yes i intend to take my rightful place as the crazy scientist in this universe xD)

Share this post


Link to post
Share on other sites

that would be nice. That would allow us to write some thing close to a swarm A.I for drone or some bad ass predictive algo for a targeter xD (Yes i intend to take my rightful place as the crazy scientist in this universe xD)

 

It has been stated that you will be able to make AI drones, and I've assumed your own targeting system as well. We have similar intentions with drone swarms and automated defenses. We are a few in number but tend to operate on the scale of a large group so we need to be able to automate things. But I'd say you're thinking to small.

 

My group has been developing a much more sinister plan. We're just waiting to see if all the features we need to pull it off will be implemented.

Share this post


Link to post
Share on other sites
One question about the constructs and DPU devices... Where will they be executing?  On the user's local machine if they are in the area, or is there a hosted cloud / server service for processing all of these DPU events?  

 

My question is largely because if run locally, the effectiveness of DPU code would be higher on machines they have stronger CPU power, giving a possible advantage to more powerful machines as they theoretically can run more DPUs at once without lagging down the game.

Share this post


Link to post
Share on other sites

 

One question about the constructs and DPU devices... Where will they be executing?  On the user's local machine if they are in the area, or is there a hosted cloud / server service for processing all of these DPU events?  
 
My question is largely because if run locally, the effectiveness of DPU code would be higher on machines they have stronger CPU power, giving a possible advantage to more powerful machines as they theoretically can run more DPUs at once without lagging down the game.

 

 

Hmm, but what if you manage to find a way to break into the DPU using third party programs or tools and have a way to manipulate the lua outside of the protection of DU servers?..

 

I understand your question, but my concern is about it being safe, I'm more than happy for it all to remain on the server if it means that nasty people can't cheat. But as we don't know the in depths parts of your question i'll leave it to Nyz to respond correctly to you.

 

If they manage to make a locked system on a local machine, what does that mean for people who don't have the greatest of machines in comparison to those who have top-end machine. They would be limited in their game play by how good their computer is. I don't know if i would enjoy that. I have a top-end machine, but i don't want to win or have the advantage simply because i'm capable of doing more lua/dpu things than other people.

 

If that makes sense. I like things to be fair, and i think having local machines would imbalance it.

 

nora,

Share this post


Link to post
Share on other sites

Hmm, but what if you manage to find a way to break into the DPU using third party programs or tools and have a way to manipulate the lua outside of the protection of DU servers?..

 

I understand your question, but my concern is about it being safe, I'm more than happy for it all to remain on the server if it means that nasty people can't cheat. But as we don't know the in depths parts of your question i'll leave it to Nyz to respond correctly to you.

 

If they manage to make a locked system on a local machine, what does that mean for people who don't have the greatest of machines in comparison to those who have top-end machine. They would be limited in their game play by how good their computer is. I don't know if i would enjoy that. I have a top-end machine, but i don't want to win or have the advantage simply because i'm capable of doing more lua/dpu things than other people.

 

If that makes sense. I like things to be fair, and i think having local machines would imbalance it.

 

nora,

 

That's why Lua interfaces are typically sandboxed objects that are being interacted with.  ie. The "Engine" DPU that you plug into isn't the actual engine, it is a proxy exposing access to the game engine DPU events and functions.  

 

This would prevent hijacking of the game engine on servers.

Share this post


Link to post
Share on other sites

I'm super excited about this! Gonna have to freshen up my knowledge of LUA in anticipation! Perhaps at some point I can work on some guides once we see how this actually works in-game.

Share this post


Link to post
Share on other sites

 

 
This above custom made Control Unit DPU could be seen as a black box, a �Stabilizer�, with 7 slots (5 engines, 1 gyro and the System DPU). You can move this black box in what we call a Component DPU, so that you can sell/exchange it. Now players possessing your �Stabilizer� can plug it in their own Control Unit DPU along with their own Elements, and then connect these Elements in the Component slots, with a simple drag and drop interface, or a smart �autoconfigure� system, making the whole process of scripting a ship very simple for those who don�t want to put their hand on code:
 
 

 

 

I am assuming this "black box" you can sell is protected by flags so another player can not open it to see your code.  Is this the case?  Also does that mean that I can give flag access to a programming partner???

Share this post


Link to post
Share on other sites

I am assuming this "black box" you can sell is protected by flags so another player can not open it to see your code.  Is this the case?  Also does that mean that I can give flag access to a programming partner???

 

I think he meant a literal black box.  As in the physical manifestation of it in the game is a box shaped object that's black... that you place inside the ship... building... drone... whatever you're making.

Share this post


Link to post
Share on other sites

If I want to create something like a magnetic field to disrupt energy weapons, do I have the option to do that out of the box or do we need to wait for the NQ devs to create a "shield" entity? Will I even be able to disrupt energy (electromagnetic) weapons using a magnetic field (as can be somewhat done with physics)? Are we getting access to a physics engine via API or a set of standard "entities"?

 

You mentioned self.engine2.setPower(42) in your example. Where does engine2 come from? Is that a component that I could have scripted in LUA or some default "entity" created by the NQ devs? If  I created it, what actions can I set? Are we accessing system defined entities like "engine" or are we accessing physics engine APIs such as velocity, thrust, torque and the like?

Share this post


Link to post
Share on other sites

I've just started exploring the devblog but I very much want to learn more about the factory side of things. 

Will the owner of the factory have to provide all of the material and be present for the factory to run?

Or will there be some mechanism where customers can drop all of the required materials into a hopper, submit payment and pop out a construct?

Perhaps this steps on the toes of the blueprint market and that's where the action will be?

 

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

×
×
  • Create New...