Jump to content

DevBlog: Lua Improvements and Changes - Discussion Thread


NQ-Deckard

Recommended Posts

3 minutes ago, FireDemon said:

Hey Guys,

 

I have tried the following code in lua:
 

list = radar.getConstructIds()

 

And the game only replies with "attempt to call a nil value (field 'getConstructIds')"

 

Could it be that these features are not implemented in the game yet?

 

Thanks for your reply,

FireDemon

'radar' here is a variable set up with a radar object.  You either need to have the autoconf system set it when your .conf file is applied or you'll need 'local radar=slot[2]' or whatever after you linkit to a particular slot.

 

Also I think getConstructIds got taken out?

 

A.

Link to comment
Share on other sites

13 minutes ago, FireDemon said:

Hey Guys,

 

I have tried the following code in lua:
 

list = radar.getConstructIds()

 

And the game only replies with "attempt to call a nil value (field 'getConstructIds')"

 

Could it be that these features are not implemented in the game yet?

 

Thanks for your reply,

FireDemon

Not LIVE yet

Link to comment
Share on other sites

On 10/8/2021 at 6:02 PM, NQ-Ligo said:

This may seem simple from an external point of view, but it is not. Indeed, we don't necessarily have the resources to rework the whole Lua API at the moment and maintain multiple versions, even if we would like to rework it completely, there are more priorities at the moment.

Anyway, be sure I've passed this point on to the rest of the team ;) . 

 

I understand, believe me (been there, done that).

 

Link to comment
Share on other sites

On 10/12/2021 at 2:56 PM, Zeddrick said:

Drop LUA. Use WASM or JS. it's 2021 not 2003.

 

EDIT: it was Kirth Gersen who I intended to quote, and not Zeddrick (I accidentally quoted a quote, and the attribution got messed up). Apologies for any confusion caused.

 

Lua isn't really the problem, and it's very lightweight and relatively easy to integrate into other apps, so it's a sensible choice.

 

The problem is everything that surrounds it - the tooling, support for external libraries, versioning, debugging, and so on - along with the richness (or not) of the in-game APIs.

 

Just switching to JS wouldn't necessarily solve those problems. You wouldn't automatically be able to use npm and just pull in any random library you wanted. There would be all sorts of implications in terms of overhead, and the logistics of making sure that every client was running exactly the same code. Most of those problems could be solved, with the right level of investment, but they could also be solved for Lua.

 

Edited by Samedi
Corrected attribution
Link to comment
Share on other sites

On 10/7/2021 at 10:30 AM, sysadrift said:

Plz don't delete quats.

 

Completely agree!!! I wish I'd known we had quaternions. I use those all the time in Second Life scripting. Much better than Euler vectors even if harder to interpret visually. Please keep them. And document them better lol.

 

JC said a long time ago that we would eventually get access to orbital parameters, which I know the game engine has because you need them to draw the trajectory line. Will we get these? I'd rather not calculate them in Lua - we don't need more sources of lag.

Link to comment
Share on other sites

15 minutes ago, Daphne Jones said:

Completely agree!!! I wish I'd known we had quaternions. I use those all the time in Second Life scripting. Much better than Euler vectors even if harder to interpret visually. Please keep them. And document them better lol.

 

JC said a long time ago that we would eventually get access to orbital parameters, which I know the game engine has because you need them to draw the trajectory line. Will we get these? I'd rather not calculate them in Lua - we don't need more sources of lag.

Calculating the orbital parameters is very cheap. No reason to add them to the API really.

Link to comment
Share on other sites

4 hours ago, Zeddrick said:


OK, so there are two things here.  Firstly, LUA in DU is not like the LUA in WOW.  In DU the lua is part of a construct which can be killed, cored and taken by someone else.  And without DRM *the person who takes the construct gets your scripts* when they kill your ship.  Or they might get the scripts belonging to whoever gave you them, etc.

However DRM doesn't really enable you to sell scripts at all (outside of certain special situations where you trust another player a lot) because it doesn't work for scripting.  The DRM owner of a script is whoever deployed the original construct and built the ship.  It's not the person who wrote the script.  So as a script writer you either become a ship builder too and sell ships or you're SOL when it comes to selling scripts because you can't have any DRM protection at all on them.  You can't even put PvP scripts you write on your own PvP ships unless you built them from scratch because you have to ask the original owner to enable DRM, then every time you want to change your own script you have to ask them to come do it again.  Or you can fly without DRM and if someone kills you then they get to take your PvP script and they can sell it to other people if they want.

 

The whole thing is miserable.  I'm waiting for NQ to do it properly before I even think about selling scripts.  Until then my scripts are for me only.
 

Thank you for explaining my joke. Glad you got it.

Link to comment
Share on other sites

1 hour ago, Daphne Jones said:

It's a lot cheaper in C and already being calculated there. Small differences matter when you're doing it every frame.

It is probably honestly not really worth it. Probably the entire calculation ends in 0.03-0.1ms. Absolutely minimal impact if done right.

Link to comment
Share on other sites

What are time frames when all these Lua changes described in devblogs will be live? Is it weeks/months/anything else? 

 

Question:

DevBlog#2 has Radar functions description. Is this the full list of functions for the Radar? I mean, now Radar has .getData() function, will it be still there when these changes will come to live?

Link to comment
Share on other sites

1 hour ago, Sawafa said:

What are time frames when all these Lua changes described in devblogs will be live? Is it weeks/months/anything else? 

 

Question:

DevBlog#2 has Radar functions description. Is this the full list of functions for the Radar? I mean, now Radar has .getData() function, will it be still there when these changes will come to live?

I’d say yes since it’s more or less a generic element function, but @NQ-Ligo can probably confirm it for us.

 

But I wouldn’t use it if I were you. I would localise the functions you intend on using and iterate over the construct list. radar.getData() causes a massive amount of lag in the game because it’s formatting the entire data set into JSON and then we decode that JSON, causing even more overhead. It’s significantly worse using it so I would definitely avoid it.

 

I did a test for this btw. That single function caused 20ms of lag, this meant it actually caused the entire frame to wait for that to complete each time, meaning what should have been 30 FPS ended up being 15 FPS. This is in a 2000 core area, true, but it goes to show that it’s stupidly costly lag-wise.

Link to comment
Share on other sites

2 hours ago, Daphne Jones said:

It would be nice if we could get planet positions and sizes too. Like a function that returns data about the most influential planet/moon atm (by gravity). Yeah, I have a script that has those hard coded too, but that's gonna be f-ing useless when the next system comes in.

 

The Atlas probably will have that data (I hope). Next dev blog will probably enlighten us more on it. 

Link to comment
Share on other sites

5 hours ago, EasternGamer said:

I would localise the functions you intend on using and iterate over the construct list.

 

Sure, the question was only just to know if all existing radar scripts would stop to work. It's OK, as having all the info in the function calls and not in the json is much easier, but it is good to  know if your code must be rewritten or it will not work at all.

Link to comment
Share on other sites

Thanks for the second part of the series.  Reading about the radar changes you seem to be adding a bunch of API calls so one can access the radar data without using the radar.getData() method.  I like this, not in the least because calling json.decode(radar.getData()) in a busy location overloads the CPU, causing the script to be killed, so using the radar data needs a bit more thought.

 

However one of the things my script (and most PvP scripts I imagine) wants to do is produce filtered subsets of the radar widget so instead of just a 'radar' box there are 'friendly', 'neutral', etc widgets which are separate.  Doing this means getting the JSON data from the radar, modifying it and adding the modified data to custom widgets with system.updateData (or whatever it's called).  In order to make the code which does this robust, my script has to decode the JSON (using a custom version of dkjson with yield statements!), manipulate the data into multiple structures, re-encode those back into JSON and update the widget data.

What this means is:
- I always have a binary copy of the radar data kicking around anyway.  This comes for free as part of the above process.  So I don't need the new calls you're adding because looking at the data tables is probably going to be faster than making the API function calls.

- The updated radar API has done nothing at all to solve the pain points when developing a PvP radar script.  I still need a complete copy of the JSON parser and have to keep wasting CPU encoding/decoding JSON constantly.

What would work better, IMO, is to have a binary version of the getData() call which returns the decoded structure and a matching updateDataFromTable (or whatever) call to allow modified versions of that raw data to be fed back into the display widgets.  That would make my script much easier to write and remove the part which takes the most CPU -- the need to constantly encode/decode JSON to make the IFF widgets.

Link to comment
Share on other sites

Hi all!

First of all, thank you for the feedback, and we apologize for the delay in publishing part 3. As my dear colleague @NQ-Deckard explained on discord, we encountered some potential problems, and so we preferred to take the time to check it out instead of announcing something and coming back to it later.

Planet positions and size ... an Atlas ?

12 hours ago, Daphne Jones said:

It would be nice if we could get planet positions and sizes too. Like a function that returns data about the most influential planet/moon atm (by gravity). Yeah, I have a script that has those hard coded too, but that's gonna be f-ing useless when the next system comes in.

 

If you let me do, I will tell you that patience is rewarded, don't worry. ?


Radar API changes:

11 hours ago, Sawafa said:

What are time frames when all these Lua changes described in devblogs will be live? Is it weeks/months/anything else? 

 

Question:

DevBlog#2 has Radar functions description. Is this the full list of functions for the Radar? I mean, now Radar has .getData() function, will it be still there when these changes will come to live?

To be concrete, we have stipulated explicitly in the change log if a function is deprecated or not. In this case, getData will remain don't worry, especially considering it is a common function for all elements.
We prefer not to give specific dates, but we will say that the third part will be published soon. ;) 


Radar API and JSON:

56 minutes ago, Zeddrick said:

Thanks for the second part of the series.  Reading about the radar changes you seem to be adding a bunch of API calls so one can access the radar data without using the radar.getData() method.  I like this, not in the least because calling json.decode(radar.getData()) in a busy location overloads the CPU, causing the script to be killed, so using the radar data needs a bit more thought.

 

However one of the things my script (and most PvP scripts I imagine) wants to do is produce filtered subsets of the radar widget so instead of just a 'radar' box there are 'friendly', 'neutral', etc widgets which are separate.  Doing this means getting the JSON data from the radar, modifying it and adding the modified data to custom widgets with system.updateData (or whatever it's called).  In order to make the code which does this robust, my script has to decode the JSON (using a custom version of dkjson with yield statements!), manipulate the data into multiple structures, re-encode those back into JSON and update the widget data.

What this means is:
- I always have a binary copy of the radar data kicking around anyway.  This comes for free as part of the above process.  So I don't need the new calls you're adding because looking at the data tables is probably going to be faster than making the API function calls.

- The updated radar API has done nothing at all to solve the pain points when developing a PvP radar script.  I still need a complete copy of the JSON parser and have to keep wasting CPU encoding/decoding JSON constantly.

What would work better, IMO, is to have a binary version of the getData() call which returns the decoded structure and a matching updateDataFromTable (or whatever) call to allow modified versions of that raw data to be fed back into the display widgets.  That would make my script much easier to write and remove the part which takes the most CPU -- the need to constantly encode/decode JSON to make the IFF widgets.

I completly see your issue Zeddrick, even if I would love to completly obliterate the JSON in Lua API and return structures, we can't do that for the moment.


Anyway, in your case, considering that we tried to provide you with access to all the information accessible in the getData JSON, you should be able to save yourself parsing and concatenation steps on the JSON as a string or decoding the JSON . You could build your structure directly on a Lua table and encode it in JSON at the end of the processing to integrate it in the widgets.
I realize that all this is not ideal though and that a structure would be better.

 

Link to comment
Share on other sites

NQ, please do not change/not delete radar.getData() so we can create custom radar widgets.

If you change getData we will have to do our own json structure, pity us...

Link to comment
Share on other sites

8 minutes ago, SneakySnake said:

NQ, please do not change/not delete radar.getData() so we can create custom radar widgets.

If you change getData we will have to do our own json structure, pity us...

As I said, we will not remove it ;) As all functions on the Radar API which didn't changed or events ...etc.

Link to comment
Share on other sites

Just now, NQ-Ligo said:

As I said, we will not remove it ;) As all functions on the Radar API which didn't changed or events ...etc.

Thank you! ?

Link to comment
Share on other sites

2 hours ago, NQ-Ligo said:


Anyway, in your case, considering that we tried to provide you with access to all the information accessible in the getData JSON, you should be able to save yourself parsing and concatenation steps on the JSON as a string or decoding the JSON . You could build your structure directly on a Lua table and encode it in JSON at the end of the processing to integrate it in the widgets.
 

Yes, I did consider doing partial parsing of the string.  However:
-- I don't like doing things like partial parsing of JSON strings because that sort of code is inherently flaky.  I know you aren't allowed to name a ship "},{constructId=XX,name=YY}" but if you could you would be able to completely break most of the regexp-and-string concatenation types of parsers people would probably make.  And this type of code isn't nescessarily all that future proof against the JSON structure changing and adding more nested data, etc.  I just like to know my code is protected against a lot of things like that.


-- I'm not just re-constructing the original JSON.  I'm also taking the construct ID, mangling it into a tag and putting it on the front of the name for each element on the list before updating the radar widget.  That would require a reasonably deep parsing of at least part of the JSON so I decided I might as well parse it all and gain the above robustness.

Building it on a table is sort of viable, but I would need to be able to build the data up completely and not all of the information is there.  For example, one of the fields tells the radar widget how far along identification is.  That isn't exposed in the new API so instead of seeing a progress bar when identifying a target people would see it jump straight to 50% (say), then wait and jump straight to 'identified'.  Also the code isn't that future-proof against new things getting added to the radar data and being used by the radar widget.

 

Thanks for the response though.

Link to comment
Share on other sites

On 10/11/2021 at 8:40 PM, Kirth Gersen said:

Drop LUA. Use WASM or JS. it's 2021 not 2003.

They're millions of people, tools (linters and ide), libraries, transpilers, etc that can produce code for WASM or JS.

 

Sure LUA is popular in gaming mainly because of World of Warcraft (WoW) addons system. But WoW is from 2003 not 2021. Back then PC were slower, with less RAM and CPU cores  and the JS ecosystem wasn't very big. LUA was a good choice back in 2003. It's not in the 2020s.

Progress have been made since then. JS came along and dominated. Now WASM it the next step. Build for the future not for 2003.

 

There are tons of UI code, physics code, atmo & space flight code, whatever code and libraries already developed. 

 

You're losing lot of skilled programmers and a huge existing base code by not using WASM for this game. Even JS is better than LUA nowadays because tons of stuff exist in JS and they're tons of transpilers to JS.

 

Hello everyone, I just want to quickly chime in on the conversation and take the time to respond to this post.

There are a few factors at play here, including security, sandboxing and learning.

Particularly the last one, as Lua is a very verbal language, with terms such as 

if something == true then 
    doAction() 
end


It is a particularly easy language for someone to learn who has never seen code before, and as Dual Universe is a game for everyone it is a much better fit.

Besides that, it also meshes well with C++, can be nicely sandboxed and is commonly used in many games and thus a familiar platform for many players.

Chances are high, that if you have good knowledge of JS, Lua will be a piece of cake to learn.

 

On top of all that, there is also simply: Lua, Lua, oh baby, said we gotta go. Yeah, yeah, yeah, yeah...
 

 

Link to comment
Share on other sites

On 10/12/2021 at 4:41 PM, Samedi said:

 

Lua isn't really the problem, and it's very lightweight and relatively easy to integrate into other apps, so it's a sensible choice.

 

The problem is everything that surrounds it - the tooling, support for external libraries, versioning, debugging, and so on - along with the richness (or not) of the in-game APIs.

 

Just switching to JS wouldn't necessarily solve those problems. You wouldn't automatically be able to use npm and just pull in any random library you wanted. There would be all sorts of implications in terms of overhead, and the logistics of making sure that every client was running exactly the same code. Most of those problems could be solved, with the right level of investment, but they could also be solved for Lua.

 

You made it look like I said something in your quote there when I actually said the exact opposite!  Be careful to quote people properly ...

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...