Overview

• Some desirable features for a MT theory
• Goals of MT
• Rule-based MT
• Statistical MT
• Where SMT is weakest
• Illustrations from a rule-based project, L³
• Hybrid models and future work

Components of an MT system

Two properties of a good translation

• It is faithful to the input (the meaning is preserved).
• Is is fluent (grammatical and easily comprehensible to a native speaker of the target language).

Other desirable features of an MT theory

• Language pair independence
• Extensibility
• Modularity (language, grammar, lexicon)
• Robustness
• (Bidirectionality)

Two kinds of cross-lingual information needs

• The user wants the gist of one or more documents.
  • The output need not be strictly faithful to the input.
  • The output may contain grammatical errors (be relatively non-fluent).
  • Processing speed probably matters.
• The user wants a translation of a document that is of publication quality.
  • The output must obey the grammatical rules of the target language.
  • The output must be faithful to the input.
  • Processing speed may not matter.

Rule-based machine translation

• Knowledge acquisition
  • Linguists or knowledge engineers write grammars and transfer rules for particular language pairs
  • Requires many person-hours of work by experts
• Processing
  • Source language input sentence, S₀, is analyzed (morphologically, syntactically)
  • On the basis of lexemes and structures in S₀, rules in the KB are accessed and applied, yielding target language sentence lexemes and structure
  • Target language sentence is generated

Rule-based machine translation

• Survives today in a number of practical systems, especially in Europe
• Desirable features
  • Extensibility: YES
  • Modularity: YES
  • Robustness: NO
  • Bidirectionality: SOMETIMES

Statistical machine translation

• Knowledge acquisition (training)
  • Requires a large corpus of sentence-aligned translation pairs.
  • Using various machine learning techniques, parameters that permit estimation of P(T|S) for a given S and T are learned
    • Words in the sentence pairs are aligned
    • Bilingual phrases are extracted
    • Word-word and phrase-phrase translation probabilities are learned
    • Various order-related parameters are learned
    • A statistical model of the target language is learned

Statistical machine translation

• Processing (decoding)
  • Search for the target language sentence T that maximizes the probability P(T|S)
  • Exhaustive search not possible, so various heuristics used to constrain the search space
• The focus of most MT research
• Behind the most popular MT system, Google Translate
• Desirable features
  • Extensibility: NO
  • Modularity: NO
  • Robustness: YES
  • Bidirectionality: NO

So which wins?

• It depends on your purpose.
• SMT has made astounding progress in the 20 years since it emerged.
• For language pairs with sizable bilingual corpora, it does a good job of extracting gist from a document.
• However, when faithfulness and fluency are important, or when sufficiently bilingual corpora are not available, it fares not so well.

Where SMT does badly

• The simplest mapping of transitive sentence arguments (Direct Transitive Mapping)

  Vs	→	Vt
  Sbjs	→	Sbjt
  Objs	→	Objt

• The doctor cured the child.
  ሐኪሙ ልጁን አዳነው።
• Other such verbs: know/አወቀ, respect/አከበረ, hit/መታ, eat/በላ, read/አነበበ, buy/ገዛ, etc.
• This pattern is extremely common, probably across all pairs of languages

How does SMT do with DTM?

• Some English-Spanish examples from Google Translate
  • The doctor cured the child. El médico curó al niño.
  • The lawyer bought a company. El abogado compró una empresa.
  • The doctor cured the woman. El médico cura a la mujer. ('cures')
  • I cured the woman. Me curó a la mujer. ('Me cured the woman.')
  • She cured her. Se curó. ('She was cured.')
  • We respect the lawyer. Respetamos el abogado. (el should be al)
  • We read the book. Al leer el libro. ('On reading the book.')
  • They are drinking the beer. Que están bebiendo la cerveza. ('That they are drinking the beer.')
  • The women knew three nurses. La mujer sabía que tres enfermeras. ('The woman *knew that three nurses.')
  • The nurse killed the lawyer. La enfermera de la muerte del abogado. ('The nurse of the death of the lawyer.')

The problem of complex morphology

• SMT systems rely on words that occur in corpora.
• When many words are possible, there is the problem of data sparsity.
  • When an unknown word appears in the input, the system can't translate it.
  • When an unknown word must appear in the output, the system can't output it.
• Languages with complex morphology have many more possible words than languages with simple morphology.

Morphology: English vs. Amharic

SMT: DTM and complex morphology

• Some English-Swahili examples from Google Translate
  • The women knew three nurses. Wanawake alijua wauguzi tatu. (two grammatical errors in 'knew')
  • The doctor cured the child. Daktari kutibiwa mtoto. ('to be cured')
  • The lawyer bought a company. Mwanasheria wa kununua kampuni. ('Lawyer of to buy company.')
  • We respect the lawyer. Sisi heshima wakili. ('We respect (noun) lawyer.')
  • They are drinking the beer. Wao ni kunywa bia. ('They are to drink beer.')

Linguistic generalizations

• There are a number of simple linguistic generalizations related to these translation problems.
  • For DTM verb pairs, subjects and direct objects map onto one another in the two languages.
  • Each language has a particular way of realizing the direct object. (For example, Amharic marks definite direct objects with the suffix -ን.)
  • The order of subject, verb, and direct object is constrained in particular ways in each language.
  • The subject may be dropped in many languages; the object may be dropped in some languages.
  • Verbs in many languages agree in person, number, and sometimes gender with their subjects.
  • Verbs in some languages agree in person, number, and sometimes gender with their direct objects.
  • Though complex, the morphological systems of many language can be described by well understood rules.

Linguistic generalizations

• SMT can only generalize on the basis of the examples it has seen and (in its basic form) cannot learn rules.
• RBMT, on the other hand, readily incorporates such generalizations.

Basic steps in RBMT

1. Morphological analysis of source language sentence
2. Syntactic analysis of source language sentence
3. Application of transfer rules
4. Syntactic generation of target language sentence
5. Morphological generation of target language sentence

An example RBMT system

• L³: a free/open-source general-purpose MT system under development
• Focus on under-resourced and morphologically complex languages
• Intended application: machine translation or computer-assisted translation with publication-quality output
• Currently completely rule-based
• Future work: integrating statistical components

A linguistic theory: Extensible Dependency Grammar (Debusmann, 2007)

• (Human-engineered) dependency grammar
• Analysis and generation by constraint satisfaction
• Flexibility, modularity, expressiveness
• Multiple dimensions: syntax, semantics, etc.

Dependency grammar and XDG

A translation sentence pair in XDG

Where the rules are

• Dependency grammar is completely word-based; the grammar is in the lexicon
• Part of the (slightly simplified) entry in the English lexicon for transitive verbs

  V_T
     syn:
        out: {sbj: !, obj: !}
     synsem:
        link: {arg1: sbj, arg2: obj}
  

• The entry for the English lexeme for like inherits from the V_T entry

  ~V~like
     classes: [V_T]
     sem:
        lex: LIKE
  

Translation in L³

• Basic steps
  1. Morphological analysis of source words (HornMorpho)
  2. Source sentence words are looked up in lexicon, constraints are instantiated (target language lexemes and constraints are accessed through the Semantic dimension)
  3. Constraint satisfaction
  4. Morphological generation of target words (HornMorpho)
• Since the constraints are (mainly) declarative, a multilingual grammar permits translation in different directions
• Some examples of translation

Current/future work on L³

• Hybrid models
• Some aspects of translation are obviously better handled by machine learning and statistics
  • Cross-lingual word-word and phrase-phrase associations
  • Disambiguation: learning weighted constraints
• Given a rudimentary XDG grammar, it may be possible to learn additional constraints or modify existing ones

References

• Debusmann, R. (2007). Extensible Dependency Grammar: A modular grammar formalism based on multigraph description. PhD Dissertation, Universität des Saarlandes.
• Gasser, M. (2010). A dependency grammar for Amharic. Workshop on Language Resources and Human Language Technologies for Semitic Languages.
• Gasser, M. (2011). HornMorpho: a system for morphological processing of Amharic, Oromo, and Tigrinya. Conference on Human Language Technology for Development, Alexandria, Egypt.
• Gasser, M. (2011). Towards synchronous Extensible Dependency Grammar. International Workshop on Free/Open-Source Rule-Based Machine Translation, 2.