# amisfilter: Tool for making amis-style data files

Japanese version

This tool makes amis-style data files from an unfiltered event file.

amisfilter model_name mask_module uevent_file count_file model_file event_file
model_name | name of a probabilistic model (also used in parsing)
mask_module | lilfes program in which masks that are applied to unfiltered events are implemented
uevent_file | input unfiltered event file (text or compressed (gz or bz) format)
count_file | file to output feature counts (text format)
model_file | output model file (AmisModel format)
event_file | output event file (AmisEvent format)
Options
-t threshold | threshold of feature frequency (default: 1)
-n threshold | limit number of outputting events
-v | print debug messages
-vv | print many debug messages

This tool makes amis-style data files from an unfiltered event file that is made by "unimaker" or "forestmaker".

"unimaker" or "forestmaker" outputs unfiltered events that are separated by "//" in the following format. The last field is a category name.

>     in//IN//vp[PPnp]//uni

Features of a maximum entropy model are generated by applying "0/1" masks to each field of a string in the above format. For example, supposing applying the mask like (0, 1, 1) to the above string, we obtain the following features.

>     _//IN//vp[PPnp]//uni

Masks are described with "feature_mask/3" defined in "amismodel.lil". A mask must be a list of 0 or 1, and must have the same length as the fields of unfiltered events.

`feature_mask(+$ModelName, -$Category, -$Mask)`
$ModelName | name of a probabilistic model
$Category | name of a category
$Mask | a list of 0 or 1 that represents a mask
Specify a mask to be applied to unfiltered events.

"amisfilter" first generates features using masks. Next, it counts the frequencies of features in observed events (empirical frequency), and outputs them into a count file. Finally, it adopts features whose frequencies are above the threshold, and makes model and event files in amis-style.

MAYZ Toolkit Manual MAYZ Home Page Tsujii Laboratory