Creating a custom scripting language
Goto page 1, 2, 3  Next
 
Post new topic   Reply to topic    mudlab.org Forum Index -> Coding
View previous topic :: View next topic  
Author Message
bullseye



Joined: 19 Jul 2005
Posts: 10
Location: Kansas

PostPosted: Sat Jul 23, 2005 4:49 am    Post subject: Creating a custom scripting language Reply with quote

I've taken up writing a codebase as a long-term hobby, and while I can integrate Python, Ruby, and probably a dozen other interpreted languages nicely into a C# environment, I am leaning heavily toward writing my own interpreter. The complexity surrounding compiler creation is not at issue here, and using the built-in C# compiler classes for dynamic code isn't really the direction I want to go in. I'm actually leaning toward a OO design like in UnrealScript, complete with a similar state/event model, but don't have aspirations for it to be quite as detailed or complex at this time.

I am primarily interested in feedback on what features the scripting language should have, what level of depth and complexity is appropriate given the end user of the language, and opinions on whether or not it is going to be difficult to convince beginning and intermediate level MUD coders to learn a new script. If anyone has already gone down this road, that feedback, good or bad, would be welcome as well.

Thanks.
Back to top
View user's profile Send private message
Author Message
Kjartan



Joined: 13 May 2005
Posts: 110

PostPosted: Sat Jul 23, 2005 7:01 am    Post subject: Reply with quote

I wrote one using lex and yacc. You can read about it here. It was written in a complete absence of knowledge of other mud scripting languages (except that I had seen PennMUSH's and I didn't like it, it was too hard to use). So far half a dozen people have become fairly proficient with it over the two years since I made it.
Back to top
View user's profile Send private message Visit poster's website
Author Message
Lindahl



Joined: 29 May 2005
Posts: 56

PostPosted: Sun Jul 24, 2005 4:19 am    Post subject: Re: Creating a custom scripting language Reply with quote

I think it really depends on what you want to use the scripting language for and who's going to be using it. I think there's a lot to be said about AI script utilized by builders that's pretty much a state-based programming model. For example, they have scripts for behavior upon entry, exit and during a state (just a rough guess of a design). There's a lot you can simplify by assuming this model, which is always a good thing, IMO, when you consider builders as the end-users. They aren't dumb in any sense, but their efforts are much better spent on the creative end.

However, you do lose flexibility and power, this way - and it doesn't work if you want to use the scripting language for a lot of gameplay elements. I know a few MUD designers are going this route. You can probably bend a more flexible langauge (i.e. Ruby - Unifex's Aetas) into a much more simple model for builders, but I have skepticism about such hacks - does anyone have experience doing anything like this?
Back to top
View user's profile Send private message
Author Message
eiz



Joined: 11 May 2005
Posts: 152
Location: Florida

PostPosted: Sun Jul 24, 2005 6:26 am    Post subject: Reply with quote

bullseye wrote:

I've taken up writing a codebase as a long-term hobby, and while I can integrate Python, Ruby, and probably a dozen other interpreted languages nicely into a C# environment, I am leaning heavily toward writing my own interpreter.


Good. It can be rather difficult to wedge things like a security model, decent performance, lightweight tasks, or any other useful feature into someone else's interpreter. Especially when you're calling across an FFI barrier like C#->C.

bullseye wrote:

The complexity surrounding compiler creation is not at issue here, and using the built-in C# compiler classes for dynamic code isn't really the direction I want to go in.


Writing a language in C# that doesn't do bytecode generation is going to come with a serious performance penalty, unless your language is going to have APL-style primitives (which is a good idea anyway). It does offer some additional flexibility, though.

bullseye wrote:

I am primarily interested in feedback on what features the scripting language should have, ...


Good support for modelling state machines and time, either via lightweight tasks (which largely precludes compiling to .net bytecode) or some other mechanism. Beyond that it's mostly up to your personal taste. Mine tends to be rather exotic, so I won't get into it. I will say, though, that Erlang has some good ideas on how to model state machines.

Another related thing to look at is the dependency/trigger features of systems like K, Cells, and spreadsheets. In K for example, you can set triggers which are called when a given variable is updated, or specify dependencies between variables (so an update to X will recalculate Y). This is a very flexible way to handle state changes. A prototype codebase I wrote once had this feature applied to properties on objects, and I enjoyed it greatly.

bullseye wrote:

...what level of depth and complexity is appropriate given the end user of the language...


I'm not sure what you mean by "depth," but it's important to keep in mind that complexity and expressive power are not correlated. You should strive to make things as simple as possible, but that doesn't mean you have to sacrifice expressiveness or computational power. Complexity is usually a sign that you've done something wrong with your language design. That said, I'd say anything that doesn't require a background in CS or math is fair game, as long as the feature makes sense.

bullseye wrote:

...and opinions on whether or not it is going to be difficult to convince beginning and intermediate level MUD coders to learn a new script.


Beginners are no problem. It's the 'power users' who have memorized 1000s of tricks they learned on their last MUD that you need to worry about. It'll probably be about as hard to convince them to learn something new and different as it would be for me to convince you to implement something new and different.
Back to top
View user's profile Send private message Visit poster's website
Author Message
Yui Unifex
Site Admin


Joined: 11 May 2005
Posts: 47
Location: Florida

PostPosted: Mon Jul 25, 2005 4:10 pm    Post subject: Reply with quote

So far I have worked on embedding Ruby, JavaScript and Python into codebases for scripting purposes.

Here are the main features I'm interested in:
1) Scripts should fit into my object model with a minimum of work. I don't like to write boilerplate. If rolling your own language, you should write a wrapper layer using the .Net reflection system to handle the dirty details of script invocation. Python for .NET is fantastic with this.

2) As Eiz noted, state machines are a very important piece. To go a bit further with his brief mention, I think a script's execution context should be easily suspendable and storable so it can wait for users to respond to choices. Builders could then implement real dialog trees with a minimum of fuss.
Back to top
View user's profile Send private message Send e-mail Visit poster's website AIM Address
Author Message
shasarak



Joined: 29 Jun 2005
Posts: 134
Location: Emily's Shop

PostPosted: Mon Aug 01, 2005 3:48 pm    Post subject: Reply with quote

Pardon me for being dim, but why is it undesirable to have custom scripting language code "compiled" to MSIL and then executed within the .NET CLR? It's a pretty good virtual machine - if you're using it anyway it seems like rather a lot of work to circumvent it and write your own.
Back to top
View user's profile Send private message
Author Message
Tyche



Joined: 13 May 2005
Posts: 176
Location: Ohio, USA

PostPosted: Mon Aug 01, 2005 5:56 pm    Post subject: Reply with quote

shasarak wrote:
Pardon me for being dim, but why is it undesirable to have custom scripting language code "compiled" to MSIL and then executed within the .NET CLR? It's a pretty good virtual machine - if you're using it anyway it seems like rather a lot of work to circumvent it and write your own.


The biggest problem is there is no security model, or more accurately, a security model foreign to the requirements of muds. You end up with the same kinds of security problems that everyone using native languages (ie. Diku) have to deal with.
Back to top
View user's profile Send private message Visit poster's website
Author Message
eiz



Joined: 11 May 2005
Posts: 152
Location: Florida

PostPosted: Mon Aug 01, 2005 7:55 pm    Post subject: Reply with quote

shasarak wrote:
Pardon me for being dim, but why is it undesirable to have custom scripting language code "compiled" to MSIL and then executed within the .NET CLR? It's a pretty good virtual machine - if you're using it anyway it seems like rather a lot of work to circumvent it and write your own.


It's not necessarily undesirable, but a few reasons I can think of:

Security: Anything you can't statically verify at compile time is going to have to be wrapped anyway. It is possible to wedge in an appropriate security system on top of the CLR if you want to, but it's probably going to be a great deal of additional work.

Tasks: You need low level control over the stacks and other VM state to do efficient lightweight multitasking. The CLR gives you threads, which are too heavyweight - megabytes of virtual memory and large context switching overhead where a lightweight task that implements, say, a simple user interaction or npc brain could be a few hundred bytes, with only a few instructions to switch tasks. I suppose you could generate code in continuation-passing style (the CLR has tail calls), but this has its own problems. Namely, the CLR ignores tail calls when crossing security contexts, and they impose a severe performance penalty even without considering the CPS overhead. It also generates some more interesting security issues.

There are probably some other reasons that I'm not thinking of. Control over the bytecode loading process might turn out to be an issue, I'm not sure. Really, it's a trade-off between performance and control.
Back to top
View user's profile Send private message Visit poster's website
Author Message
BobTHJ



Joined: 19 Nov 2005
Posts: 31

PostPosted: Wed Dec 07, 2005 6:40 pm    Post subject: Reply with quote

I know I'm joining this topic rather late but...

I wrote my own LISP-like scripting language in VB.NET that I use for my mud engine, mud client, and several other projects I'm working on. Currently, it reads in a script as a string, parses it, executes various functions which can be set up at various "scope" levels, and spits out a return string. It is slow (which hasn't been a problem so far) but I've built it so that at some point in the future I can write a "compiler" that will read in a script and generate a VB.NET class file from it, which can then be compiled by .NET into bytecode. I figure that adding this extra layer of "compilation" should plug any security loopholes (assuming the script compiler writes good code). Of course, I could be way off....
Back to top
View user's profile Send private message Send e-mail
Author Message
eiz



Joined: 11 May 2005
Posts: 152
Location: Florida

PostPosted: Wed Dec 07, 2005 6:57 pm    Post subject: Reply with quote

BobTHJ wrote:
I figure that adding this extra layer of "compilation" should plug any security loopholes (assuming the script compiler writes good code). Of course, I could be way off....


A compiler would be no more or less secure than the interpreter. Of course, compiling to .net bytecode offers its own variety of problems related to code reloading (short story: thanks to the 'wonderful' design of the CLR, you can't unload assemblies, only application domains, which is a huge mess). I'd stick with the interpreter and just migrate performance critical code to VB.NET.
Back to top
View user's profile Send private message Visit poster's website
Author Message
BobTHJ



Joined: 19 Nov 2005
Posts: 31

PostPosted: Wed Dec 07, 2005 7:41 pm    Post subject: Reply with quote

I am aware of the app-domain nightmare. I guess I used the wrong term. What I've considered is less of a compiler and more of a "migration tool" that will take a script and generate VB.NET code. This VB.NET code would then be compiled into the primary mudlib assembly (or possibly a separate scripting assembly) when it is next re-compiled. I don't think I'd want to tackle dynamic on-the-fly compilation of scripts because of the appdomain issue.

My thought process was this:
1. Builder writes a script (text file)
2. The MUD loads the script using the interpreter and runs it
3. When the MUD is next compiled (next restart), the "migration tool" first writes a VB.NET version of the script, where it is compiled into the mudlib
4. The script is now in bytecode where it will execute faster on the MUD
5. The builder changes the original script file
6. The MUD recoginzes it's changed and sets a flag, causing it to load the script file using the interpreter instead of the compiled VB.NET class
7. When the mud is compiled again, the "migration tool" recognizes the script file has been modified and rewrites the VB.NET code that is compiled into the mudlib, allowing it once again to be run as bytecode.

It's far from a perfect solution, and I don't know if I'll ever actually implement it. At the present, it's not needed. Although the script interpreter is fairly slow, the mud doesn't rely heavily on scripts. This is because my mudengine provides compiled "property" objects which are essentially little bits of mud-related code that can be attached to ingame objects to give them similar function to a script. Property objects can be added to my MUD objects either from the mud command line or via XML files that the builder can edit and load into the mud. Therefore, 95% of the function that would require a script can be avoided by simply adding one or more property objects to the MUD object in question.

It's actually my MUD client that relies heavily on the scripting language. The client uses "plug-ins" to provide various enhanced function. Each plugin uses scripts that can be altered by the user to determine it's behavior. The long story:

I've abandon telnet protocol for my mud. Instead, the server sends XML messages to the client through a TCP connection. These messages include the message text (which itself may include HTML markup) that should be displayed to the client. They also include meta-data about the message. For example, a message about an attack may include the ID of the attacker, target, and weapon, the amount of damage, the type of damage, etc. The client then uses scripts to read this XML and process it as desired. So for example, the main display plugin has a script that color-codes the messages for display based on their meta-data (all color coding occurs on the client, allowing different players to set up different color-coding preferences). Another example is a script that looks for messages describing a character's inventory and launches a special inventory display plugin that gives users a graphical interface to their inventory.

Now, I'm still in development on both the client and mud engine, so I may run into trouble in the future. However, right now, even using the interpreter, the client runs at a reasonable speed displaying messages to the user only a few milleseconds after they are received.

So, long story short, I don't know if I will ever have the need to convert scripts into bytecode.
Back to top
View user's profile Send private message Send e-mail
Author Message
MikeRozak



Joined: 27 Nov 2005
Posts: 15
Location: Australia

PostPosted: Thu Dec 08, 2005 3:55 am    Post subject: Reply with quote

My suggestion about a custom scripting language is to design one that's fine-tuned to features that virtual worlds need, such as multiple inheritence. You might want to search through Richard Bartle's description of his language, especially with its relationship to parsers. Also look at Inform and Tads.

I tried to keep my syntax similar to Java, Flash, C++ since more people know those. (Richard Bartle's seems more like prolog, although I have very little knowledge of prolog.) However, I added a number of features that are designed for virtual worlds, especially for object maintinence.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Author Message
Teelf



Joined: 12 May 2005
Posts: 21
Location: Seattle, WA

PostPosted: Thu Dec 08, 2005 4:44 am    Post subject: Reply with quote

I agree about supporting multiple inheritance. MUDs naturally lend themselves to a large object hierarchy that really needs to use MI. Bartle's language is good way to handle large object hierarchies because it uses multimethods/multiple dispatch. Which makes it not like prolog, but like CLOS/Lisp.

Instead of starting a scripting language from scratch, I added multiple dispatch to Lua and have been very happy with that.
Back to top
View user's profile Send private message Send e-mail Visit poster's website AIM Address
Author Message
shasarak



Joined: 29 Jun 2005
Posts: 134
Location: Emily's Shop

PostPosted: Sun Dec 11, 2005 8:31 am    Post subject: Reply with quote

Multiple inheritance always makes me uncomfortable. It does have legitimate uses, but I find 90% of the times I initially find myself wishing I had access to multiple inheritance it subsenquently turns out that I've cocked up my single-inheritance class hierarchy.

There are two areas where people tend to wrongly think that MI is useful. The first is if they're used to working in C++. C++ is severely broken in terms of its type-checking: if you have a function into which you need to be able pass instances of two or three different classes, all of those classes have to have a common superclass so you can declare the argument as being of that type. This is not, IMO, a desirable feature, it's just a tiresome process one has to go through to compensate for compiler deficiencies. (Even if you favour static typing, a method like C# "interfaces" is much less clumsy).

The other problem is that people think of inheritance as having to deal with all of an object's capabilities and properties: in MUD terms, they expect something that a player would describe as "an object" to have a single program-object that handles all of its behaviour.

Consider a sword that has a map etched into the blade, and can glow with light, and is alive and intelligent and can speak. People think "well, this object must inherit from classes Map, LightSource, Monster, and Weapon". That actually isn't the right way to do it. You can see why if you think about how the sword behaves when it isn't glowing: when it's "on" it is a light source, when "off" it isn't, and anything checking illumination levels can ignore it.

The way to handle an object that is a light source is not to have it inherit from a LightSource class, but to create a separate object that is an instance of LightSource, and assign it as a property of the main object. (The LightSource instance not an object in terms of what a MUD player would perceive as a "game object", but in OO programming terms it is an object).

To a degree, multiple inheritance and this method (which, depending on the details, may be regarded as "containment" or "aggregation") are equivalent; the advantage is that creating new program-objects to model different aspects of a single MUD "object" is a) simpler, and b) dynamic. Languages may allow classes to change their position in the inheritance tree dynamically, but relatively few allow different objects to inherit or not inherit from different classes at different times, and this is something that's very useful in a MUD context.
Back to top
View user's profile Send private message
Author Message
eiz



Joined: 11 May 2005
Posts: 152
Location: Florida

PostPosted: Sun Dec 11, 2005 11:29 am    Post subject: Reply with quote

shasarak wrote:
Multiple inheritance always makes me uncomfortable.


Yeah, well... you're a Smalltalk user. Razz

shasarak wrote:

There are two areas where people tend to wrongly think that MI is useful. The first is if they're used to working in C++. C++ is severely broken in terms of its type-checking: if you have a function into which you need to be able pass instances of two or three different classes, all of those classes have to have a common superclass so you can declare the argument as being of that type. This is not, IMO, a desirable feature, it's just a tiresome process one has to go through to compensate for compiler deficiencies. (Even if you favour static typing, a method like C# "interfaces" is much less clumsy).


The flaws in C++'s version of multiple inheritance actually arise mostly from its interaction with pointers. The real problem is that in a lot of OO languages, subtyping assumes the role that is generally assigned to parametric polymorphism and especially existential quantification (modules). In modern C++ you will often see templates used instead, which would be a step up if C++ actually supported bounded quantification (concepts).

shasarak wrote:
The way to handle an object that is a light source is not to have it inherit from a LightSource class, but to create a separate object that is an instance of LightSource, and assign it as a property of the main object. (The LightSource instance not an object in terms of what a MUD player would perceive as a "game object", but in OO programming terms it is an object).


Which works fine until you want to have an object (let's call it a 'behavior' object since emitting light is basically such a thing) which significantly alters the function of the object that it's mixed into. Then you're going to end up with all sorts of hooks and manual delegation flying around everywhere.

shasarak wrote:
To a degree, multiple inheritance and this method (which, depending on the details, may be regarded as "containment" or "aggregation") are equivalent; the advantage is that creating new program-objects to model different aspects of a single MUD "object" is a) simpler, and b) dynamic. Languages may allow classes to change their position in the inheritance tree dynamically, but relatively few allow different objects to inherit or not inherit from different classes at different times, and this is something that's very useful in a MUD context.


Well, if we're going to talk about "the right way" as if there were a single right way (now if you want to see religious conviction, talk to an APLer), I'd have to say that prototypes and delegation are a better fit to MUDs than classes and instances any day. And of course any proper prototype based language supports multiple, dynamic delegates...
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    mudlab.org Forum Index -> Coding All times are GMT
Goto page 1, 2, 3  Next
Page 1 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

Powered by phpBB © 2001, 2002 phpBB Group
BBTech Template by © 2003-04 MDesign