---RPG Maker LcfMapUnit---
Introduction
Waah! I just totally deleted the 36-page file I was working on! And it’s all my fault! Oh well, time to start over. Brevity is now the soul of… this paper.
The Map File
The map file is composed of four sections, with a “size” seperator in between. This is beneficial; one can, say, skip straight to the events section using nothing more than a simple hand-coded lexer.
<lmu_map_file> ::= <file_type> <initial_properties> <size_hint_lower> <lower_map_tiles> <size_hint_upper> <upper_map_tiles> <size_hint_event> <event_section> <footer_section>
The File Type
The file type consists of a length hint, followed by an (ascii) string describing the content of that file. For .lmu files, this is “LcfMapUnit.” Consequently, this is one of the things Rast’s plugger modifies to “protect” a project.
<file_type> ::= <str_len><ascii>*
The Initial Properties
Initial properties of a map include its background, tileset, and scroll option. Anything with a “parent map” option (e.g. background music, teleport enable) are not represented here, neither is the map name. Since this section has no size_hint before it, we can’t skip it when parsing the map. Hence, RPG Maker takes several measures to ensure that this section is as small as possible. In particular, most commands have defaults that are simply not included in the file if they are not altered. Hence, some knowledge is implicit. (For the next few sections, imperials in italics are optional properies.) Of course, there are some dependencies here (autoscroll cannot appear without pan being checked), but since RPG Maker stores old values (i.e. speed) even when a box isn’t checked, it’d be more trouble than it’s worth to determine these, and far too inefficient to lex them. Note that, unless otherwise noted, productions/tokens undefined in a code box are assumed to be rpg_ints, which are described a few sections from now.
<initial_properties> ::= <long_property>+
<long_property> ::= <imperial><arg_len><arg_value>
<imperials> ::= (01,Chipset) , (02,Width) , (03,Height) , (0B,ScrollType) , (1F,UseParallelBkgrd?) , (20,BkgrdFileName) , (21,HorizPan?) , (22,VertPan?) , (23,HorizAuto?) , (24,HorizAutoSpd) , (25,VertAuto?) , (26,VertAutoSpd)
The Lower Tiles
The Lower tiles consist of the size_hint, followed by that many bytes. Two bytes comprise one map tile. The lower tiles are a bit complex, as the data for the derived tile is shown, not the logical tile. This causes no problems for the lower tiles that aren’t dynamic (i.e., rocks, wood, or any of the upper tiles.) However, for tiles such as grass, sand, and water, the logical tile (the one you select in RPG Maker’s map editor) differs greatly from the physical one (the actual 16x16 square shown to you in the editor screen.) Hence, whilst placing a table tile in the middle of a set of rug tiles will only change that tile’s value, placing it in the middle of a patch of ocean tiles will change its value and the values of the 9 adjacent tiles. (Note, again, that the grammar I’m producing allows maps that cannot possibly be created in RPG Maker. For instance, <tile>* in the grammar below is completely impossible; no map can have fewer than 20x15=600 tiles.)
<size_hint_lower> ::= <imperial:47><value>
<lower_tiles> ::= <tile>*
<tile> ::= <byte><byte //e.g. “05 3F”
RPG Maker’s Integer Datatype (≥ 0)
rpg_int binary* actual_value
1E 00011110 30
14 00010100 20
7F 01111111 127
81 00 10000001 00000000 128
84 58 10000100 01011000 600
FF 7F 11111111 01111111 16383
81 80 00 10000001 10000000 00000000 16384
81 AE 10 10000001 10101110 00010000 22288
BD 84 3F 10111101 10000100 00111111 999999
* black values are directly converted to decimals. (1=1, 10=2, etc.)
red values are flags denoting “multiply by 128” if true.
blue values start counting at 128 (1=128, 10=129, etc.)
green values are flags indicating, if true, to “multiply by 16384”
purple values start counting at 16384.
Note that the size_hint above has no length_hint. This is because RPG Maker uses the first bit of each integer to designate if that integer will continue. This allows one to create arbitrarily long integers without explicitly stating their length from the get-go. In all honesty, I’m not sure why .lmu files provide length_hints at all; size_hints for strings make sense, and num_arg_hints for functions are a must, but the so-called “rpg_int”s provide their own built-in length hints. It’s a compact system, for positive numbers at least. Enough theory, here’s some examples:
The Upper Tiles
The upper tiles are stored in a manner similar to the lower ones, except without the complication of derived versus logical tiles. The two sections even have the same size_hint, although the imperial is different.
<size_hint_upper> ::= <imperial:48><value>
<upper_tiles> ::= <tile>*
The Event Settings
Here’s where it gets a bit finnicky. The events section basically consists of a size_hint, followed by a number of events. Going down the line, events have their own settings, and a number of pages. Each page has its own settings, and a number of event_commands. These in turn have settings, and two (move and shop) have item or route lists. A few also allow event commands to be placed inside themselves! Size_hints are useful here, as are depth_hints. But we’re getting ahead of ourselves.
<size_hint_events> ::= <imperial:51><value>
<event_section> ::= <num_hint><event>*
Events
Each event consists of an id number, followed by some properties of the same form as map events. Following this, there are an arbitrary number of pages.
<event> ::= <id> <long_property>+ <page_bytes> <num_pages> <page>+
<imperials> ::= (01,EventName) , (02,posX) , (03,posY) , (05,size_hint_pages)
<page_bytes> = <size_hint_pages> <rpg_int>
Pages
Pages are composed of a number (similar to an id), some conditions which this event must satisfy, some graphical, activation, and other properties of the event, a route that it moves by, two (similar) size_hints, and some commands. It terminates with a page_end character. Presumably, this was included origionally to test map files; the size_hint renders this token unneccessary (although it would help a simple lexer search for, say, the number of event_commands without extending the lexer. Note that there are other ways of interpreting the data in .lmu files. In general, I have treated these map files as imperial-based data structures. This viewpoint, together with size hints, has helped to clarify a lot of “lost” bytes that would otherwise make no sense. Moreover, using this method, imperial numbers line up nicely. It was quite obvious when many of my initial guesses started at 02 that I was doing something wrong. The main contestable area which remains is that some length_hints could be construed as num_arg_hints. In general, though, this makes little sense. What kinds of imperials would express multiple properties. The obvious answer is “event_commands”; thus, num_arg_hints are used for these cases. A few other cases (one’s route) also require this distinction.
<page> ::= <number> <size_hint_ec> <event_conditions> <event_properties> <route> <size_hint_long_ecmd> <size_hint_short_ecmd> <event_command>* <page_end:00 00 00 00 00>
<size_hint_ec> ::= <imp:02><size_bytes>
<event_conditions> ::= <long_property>+ <end_ec>
<imperials> ::= (01,BoxesChecked) , (02,valSwitchA) , (03,valSwitchB) , (04,valVariable) , (05,valVarCmp) , (06,valItemCmp) , (07,valHeroCmp) , (08,valTimerCmp)
<end_ec> ::= 00
<event_properties> ::= <long_property>+
<imperials> ::= (15,pageGrahpic) , (16,pageGraphicTile) , (17,unknown*) , (19,semi-transparent) , (1F,moveType) , (20,moveFrequency) , (21,startConditions) , (22,layer) , (23,parallel) , (24,animType) , (25,moveSpeed)
<route> ::= <imp:29> <size_hint_r> <num_moves_long>? <num_moves_short> <move>* <repeat_moves>? <ignore_illegal_moves>? <null>
/* Note: num_moves tell you how many moves there are, so the <null> isn’t included. Size_hint_r, however, tells you how many bytes are left in <route>, so it includes the final <null>. Logically, they should be lexed like the global map file is. We lex them together because it is unlikely that this section will ever need to be skipped. */
<num_moves_long> ::= <imp:0B><length_hint><value>
<num_moves_short> ::= <imp:0C><value>
<move> ::= // See next paragraph.
<size_hint_long_ecmd> ::= <imp:33><length_hint><value>
<size_hint_short_ecmd> ::= <imp:34><value>
<repeat_moves> ::= <imp:15><length><value>
<ignore_illegal_moves> ::= <imp:16><length><value>
* Note: I’m still trying to figure out what this does. Also, does anyone know what happened to imperial 18? This isn’t so worrisome; the developers may have dropped a feature further into development. And thoughts on this can be sent to:
Sorlok_Reaves@yahoo.com
Route Commands
These commands are actually duplicated in the “move event” event_command; not surprising, since RPG Maker pulls up that very window to set an event’s route. Most consist of a one-byte imperial, but some may have additional arguments. Check the lmu.sable file provided for a full specification; several examples are given below.
<step_up> ::= 00
<turn_quarter_right> ::= 10
<move_frequency_down> ::= 1F
<transparency_up> ::= 28
<switch_on> ::= 20 <switch_ref>
<switch_off> ::= 21 <switch_ref>
<chage_grapihc> ::= 22 <length_hint> <ascii>+ <rpg_int: chip_id>
<play_sound> ::= 23 <length_hint> <ascii>+ <rpg_int: volume> <rpg_int: tempo> <rpg_int: balance>
Event Commands
This is where things get a bit confusing. First of all, the long_property values you’ve seen so far are no longer used. Rather, each value now takes the general form:
<event_command> ::= <imp> <depth> <str_len> <ascii>* <num_args> <rpg_int>* <extra_stuff>?
In this scheme, the imperial serves the same identification purpose it did for long_properties —except that it consists of two bytes. The depth of an argument is its nesting level; most depths are usually 00 (the default) or 01 (for, say, a “Show Choice”’s alternatives, or elements in a “Cycle.”) With Cycles within Forks, Show Choices within “Successful” branches of Inns, and other nonsense, a command’s nesting level increases linearly. The string length argument is generally 00, but can be set if certain commands (“Set hero name”, “Show picture”, etc.) require an outside reference or string input. Next comes a num_args_hint, followed by (surprise surprise) a number of arguments. This is almost redundant, since no “compression” of arguments takes place —any event_command will always have the same number of arguments. The exceptions are the “Move Event” and “Call Shop” commands, both of which take a varying number of moves/items. Finally, some extra_stuff may appear tacked onto the end of this command for, say, multiple-line message boxes, or anything with a branch. These aren’t of uniform structure for all event commands, so they need to be dealt with on a case-by-case basis. And, since we’re lexing all ec’s into their own unique nodes on the AST, we’ll do just that!
Sample Event Commands
There are a lot of event commands, and listing them all here is pointless (many are similar in style and differ only in their imperials.) Rather, we shall merely list several standard and non-intuitive ones.
<show_message> ::= <imp:CE,7E> <depth> <str_len> [line1]:<ascii>* <num_args> <additional_line>*
<additional_line> = 81 9D 0E <depth> <str_len> <ascii>* <null>
<select_face> ::= <imp:CF,12> <depth> <str_len> [gfx_file]:<ascii>+ <num_args:3> <chip> <side> <flipped>
<show_choice> ::= <imp:CF,1C> <depth> <str_len> [full_title]:<ascii>* <num_args> <default_choice> <choice_opt>*
<choice_opt> ::= <imp:81,9D> <type(2C|2D)> <depth> <str_len> [case_title]:<ascii>* <num_args> <choice_num>? <event_command>* <continue>? <case_closed:00,00>
<continue> ::= <imp:0A> <value>
<input_number> ::= <imp:CF,26> <depth> <str_len> <num_args:2> <num_digits> <variable_num>
<change_variable> ::= <imp:CF,6C> <depth> <str_len> <num_args:7> <referrent> <range_low> <range_high> <set_option> <reference> <value> <misc_option>
/* The “referrent” option shows up quite often. Here, it denotes {00:one, 01:range, 02:variable} */
<take_damage> ::= <imp:D2,04> <depth> <str_len> <num_args:8> <target_reference> <target_id> <attack> <def%> <mind%> <variance> <get_opt> <get_value>
<start_combat> ::= <imp:D3,56> <depth> <str_len> <background_file>? <num_args> <enemy_group_reference> <enemy_group_id> <battle_bkgrd_opt> <escape_case_opt> <defeat_case_opt> <first_strike_opt> <general_branch>*
<general_branch> ::= <imp:81:A1> <type> <depth> <str_len> <num_args> <event_condition>* <continue>? <case_closed>?
<call_shop> ::= <imp:D3,60> <depth> <str_len> <num_args> <shop_type> <deal_success_opt> <msg_style> <item_id>* <general_branch>?
<move_event> ::= <imp:DB,42> <depth> <str_len> <num_args> <eventID> <frequency> <repeat?> <ignore_impossible?> <move>*
<fork_event> ::= <imp:DD,6A> <depth> <str_len> [argument_string]:<ascii>* <num_args:06> <fork_conditions> <else_opt> <fork_branch>+
<fork_conditions> ::= <fcond_switch> | <fcond_variable> | <fcond_timer> | <fcond_money> | <fcond_item> | <fcond_hero> | <fcond_event> | <fcond_vehicle> | <fcond_decision_key> | <fcond_bgm_done>
<fork_branch> ::= (<imp:81,AB><type><depth><str_len><num_args>)? <event_command>* <continue>? <case_closed>?
<case_continue> ::= <imp:0A 01> <nulls: 00 00>
<cycle> ::= <imp:DF,32> <depth> <str_len> <num_args:0> <cycle_branch>+
<cycle_branch> ::= (<imp:81,AD><type><depth><str_len><num_args>)? <event_command>* <continue>? <case_closed>?
<comment> ::= <imp:E0,7A> <dpeth> <str_len> [line1]:ascii+ <num_args:0> <comment_line>*
<additional_line> ::= <imp:81,AF> <type> <depth> <str_len> [text]:ascii+ <num_args:00>
A note about event IDs:
These babies use the special character “CE”, followed by another byte, to designate the “special” map events: {11:Hero, 12:Boat, 13:Ship, 14:Airship 15:this_event}. This begs the question, then, of why we have “Set Vehicle Place” and “Goto Memorized Place” as commands at all, especially since “Set Event Place” had to specifically disable the Hero, Boat, Ship, and Airship options. On a slightly related note, this means that you can’t have more than 10,000 events on any one map. Rast, I’m sure, ran into this problem quite often in Dragon Destinies I and II. Oh, and while we’re at cutoff points, the highest integer RPG Maker can recgnize (with three digits) is 2,097,151. Fortunately for us, numbers don’t go that high, so this, also, isn’t a major issue.
A Possible Inadequacy?
One could make the argument that, technically, the imperial of “scope” is 81, and that AB, AD, etc. denotes the “kind” (fork, cycle, etc.), followed by the “type” (for “Show Choice” events, mainly.) We, however, will gloss over this, since it’s not such a big deal.
Fork Conditions
Fork conditions are a bit messy, since variables are multiplexed.
<fcond_switch> ::= <imp:00> <switch_number> <switch_value> <garbage> <garbage>
<fcond_variable> ::= <imp:01> <variable_number> <reference> <compare_to> <compare_by>
<fcond_timer> ::= <imp:02> <time_sec> <compare_by> <garbage> <garbage>
<fcond_money> ::= <imp:03> <money_amt> <compare_by> <garbage> <garbage>
<fcond_item> ::= <imp:04> <item_ID> <has_opt> <garbage> <garbage>
<fcond_hero> ::= <imp:05> <hero_ID> <hero_check>
<hero_check> ::= <hc_in_party> | <hc_name_is> | <hc_level_abv> | <hc_hp_abv> | <hc_skill_has> | <hc_equipped_with> | <hc_cond_is>
<hc_in_party> ::= <imp:00> <garbage>
<hc_name_is> ::= <imp:01> <garbage>
<hc_level_abv> ::= <imp:02> <value>
<hc_hp_abv> ::= <imp:03> <value>
<hc_skill_has> ::= <imp:04> <skill_ID>
<hc_equipped_with> ::= <imp:05> <item_ID>
<hc_cond_is> ::= <imp:06> <cond_ID>
<fcond_event> ::= <imp:06> <event_id> <face_dir> <garbage> <garbage>
<fcond_vehicle> ::= <imp:07> <vehicle_id> <garbage> <garbage> <garbage>
<fcond_decision> ::= <imp:08> <garbage> <garbage> <garbage> <garbage>
<fcond_bgm_done> ::= <imp:08> <garbage> <garbage> <garbage> <garbage>
A Second Possible Inadequacy?
Another thing you might have noticed is how stand-alone the “additional_line” tag for comments looks. One might even conclude that, since it has its own pseudo-imperial, that it should be parsed as its own command. I think it should not, mainly because:
1) It has a type, and no other event_commands do.
2) You can’t have a stand-alone “line 2” of a comment —it must follow a traiditonal comment command.
3) This cannot be done with similar event_commands such as “Fork”, which don’t have the imperial for the first case.
Improvements
This particular grammar is very, very space in-efficient. Essentially, we're saving as much information as possible, useful or otherwise. As soon as this parser is fully debugged, one will be provided which takes efficiency into consideration and saves the bare-bones minimum for each production. This includes trashing all length hints and unneccesary options, such that a production contains other (sub) productions only when the number of sub_productions is indeterminate. For example:
<map_properties> ::= <chipset>? <width>? <height>? <scroll_type> ...,
with these custom tokens picked out by our custom lexer. Note that we'd need to introduce a new token, list_end, for the "call shop" and "move event" commands.
So, basically, this parser creates a concrete syntax tree, but a future parser
will utilize our custom lexer to bulid an abstract syntax tree. The first
is more accurate to how .lmu files appear; the second is faster and takes up
less memory to store (and brain power to think about.) It is a bit uncommon for a lexer to influence how the abstract syntax tree is formed, but in this case the simplicity of RPG Maker’s map files allows for this small optimization.
A note on EBNF-RS
This should really have gone at the start of the report, but things lined up so nicely and, besides, if I’d tidied up my grammar I wouldn’t have needed this paragraph. Basically, the syntax used here is a very informal variant of EBNF notation. The main things you may not have already figured out are:
1) <random> ::= <chipset>? <ec>* <footer>+ means that the “random” production consists of zero or one chipsets, zero or more ec’s, and one or more footers.
2) <random2> ::= <opt> | <nop> means that the “random2” production contains either an “opt” or a “nop” token/production.
3) <imp:05,BC> is just a convenient way of saying “imp is the hex string 05BC”
4) <random3> ::= [options]:<list_item>+ The only thing you might not recognize here is the [options]: tag. This means that the result of list_item is referred to as “options.” I use this notation when it’s not clear what, say, “ascii+” should be considered.
...
Zitat von Dhan
CherryShare | Patches und Tools | Programmwunschthread | www.cherrytree.at | Cherry = CherryDT
Berechtigungen
Zitieren
