DIALOG Task

The DIALOG task is carried out using the Spoken Language Databases (SLDB) corpus, a collection of human-mediated cross-lingual dialogs in travel situations. In addition, parts of the BTEC corpus (see below) are also provided to the participants of the DIALOG Task.

DIALOG Training Corpus:

  • TXT data format:

    • each line consists of three fields divided by the character '\'
    • sentences consisting of words divided by single spaces


      format: <SENTENCE_ID>\01\<MT_TRAINING_SENTENCE>
      • Field_1: dialog ID
      • Field_2: sentence ID
      • Field_3: MT training sentence
    • example:

      TRAIN_00001\This is the first training sentence.
      TRAIN_00002\This is the second training sentence.
  • INFO data format:

    • each line consists of three fields divided by the character '\'
    • sentences consisting of words divided by single spaces


      format: <SENTENCE_ID>\01\<SPEAKER_TAG>
      • Field_1: dialog ID
      • Field_2: sentence ID
      • Field_3: speaker annotations ('a': agent, 'c': customer, 'i': interpreter)
    • example:

      train_dialog01\01\a
      train_dialog01\02\i
      train_dialog01\03\a
      ...
      train_dialog398\20\i
      train_dialog398\21\i
      train_dialog398\22\c
  • Languages:

    • Chinese-English (CE)
    • English-Chinese (EC)

      • 394 dialogs, 10K sentences from the SLDB corpus
  • Corpus specifications:

    • coding:UTF-8
    • word segmentations according to ASR output segmentation
    • text is case-sensitive and includes punctuation

DIALOG Develop Corpus:

  • ASR output (lattice, N-BEST, 1-BEST), correct recognition result transcripts (text), reference translations of SLDB dialogs
  • Data format:

    • 1-BEST

      • each line consists of three fields divided by the character '\'
      • sentences consisting of words divided by single spaces


        format: <SENTENCE_ID>\01\<RECOGNITION_HYPOTHESIS>
        • Field_1: sentence ID
        • Field_2: paraphrase ID
        • Field_3: best recognition hypothesis
      • example (input):

        IWSLT09_CT.devset_dialog01_02\01\best ASR hypothesis for 1st utterance
        IWSLT09_CT.devset_dialog01_04\01\best ASR hypothesis for 2nd utterance
        IWSLT09_CT.devset_dialog01_06\01\best ASR hypothesis for 3rd utterance
        ...
    • N-BEST

      • each line consists of three fields divided by the character '\'
      • sentences consisting of words divided by single spaces


        format: <SENTENCE_ID>\01\<RECOGNITION_HYPOTHESIS>
        • Field_1: sentence ID
        • Field_2: N-BEST ID (max: 20)
        • Field_3: recognition hypothesis
      • example (input):

        IWSLT09_CT.devset_dialog01_02\01\best ASR hypothesis for 1st utterance
        IWSLT09_CT.devset_dialog01_02\02\2nd-best ASR hypothesis for 1st utterance
        ...
        IWSLT09_CT.devset_dialog01_02\20\20th-best ASR hypothesis for 1st utterance
        IWSLT09_CT.devset_dialog01_04\01\best ASR hypothesis for 2nd utterance
        ...
    • reference translations

      • each line consists of three fields divided by the character '\'
      • sentences consisting of words divided by single spaces

      • format: <SENTENCE_ID>\<PARAPHRASE_ID>\<REFERENCE>
        • Field_1: sentence ID
        • Field_2: paraphrase ID
        • Field_3: reference translation
      • example:

        IWSLT09_CT.devset_dialog01_02\01\1st reference translation for 1st input
        IWSLT09_CT.devset_dialog01_02\02\2nd reference translation for 1st input
        ...
        IWSLT09_CT.devset_dialog01_04\01\1st reference translation for 2nd input
        IWSLT09_CT.devset_dialog01_04\02\2nd reference translation for 2nd input
         ...
  • Languages:

    • Chinese-English

      • IWSLT05 testset: 506 sentences, 16 reference translations (read speech)
      • IWSLT06 devset: 489 sentences, 16 reference translations (read speech, spontaneous speech)
      • IWSLT06 testset: 500 sentences, 16 reference translations (read speech, spontaneous speech)
      • IWSLT08 devset: 245 sentences, 7 reference translations (spontaneous speech)
      • IWSLT08 testset: 506 sentences, 7 reference translations (spontaneous speech)
      • IWSLT09 devset: 10 dialogs, 200 sentences, 4 reference translations (spontaneous speech)
    • English-Chinese

      • IWSLT05 testset: 506 sentences, 16 reference translations (read speech)
      • IWSLT08 devset: 245 sentences, 7 reference translations (spontaneous speech)
      • IWSLT08 testset: 506 sentences, 7 reference translations (spontaneous speech)
      • IWSLT09 devset: 10 dialogs, 210 sentences, 4 reference translations (spontaneous speech)
  • Corpus specifications:

    • coding:UTF-8
    • text is case-sensitive and includes punctuation

DIALOG Test Corpus:

  • ASR output data format: → seeDIALOG Develop Corpus
  • INFO data format: → seeDIALOG Training Corpus
  • languages:

    • Chinese-English

      • progress testset: 27 dialogs, 405 sentences of the IWSLT 2009 DIALOG task
      • IWSLT10 testset: 37 dialogs, 532 unseen sentences from the SLDB evaluation corpus
    • English-Chinese

      • progress testset: 27 dialogs, 393 sentences of the IWSLT 2009 DIALOG task
      • IWSLT10 testset: 37 dialogs, 453 unseen sentences from the SLDB evaluation corpus
  • corpus specifications:

    • coding:UTF-8
    • text is case-insensitive and does not includepunctuation