fn_spacy_tokens_for_baseline: The US State Department claims that North Korean officials admitted to having such a program during a second day of meetings with Kelly , but North Korea now claims that it has only admitted to having a " plan to produce nuclear weapons , " which Pyongyang claims is part of its right to self - defense .

source_bert_toks: the us state department claims that north korean officials admitted to having such a program during a second day of meetings with kelly , but north korea now claims that it has only admitted to having a " plan to produce nuclear weapons , " which p ##yon ##gy ##ang claims is part of its right to self - defense .

reference_raw: According to the U.S. State Department, North Korean officials admitted that they had such a program on the second day of their sessions with Kelly, but North Korea now claims that they only admitted that they had a plan to produce nuclear weapons, which Pyongyang claims is part of their right to self-defense.

reference_spacy_tokens: According to the U.S. State Department , North Korean officials admitted that they had such a program on the second day of their sessions with Kelly , but North Korea now claims that they only admitted that they had a plan to produce nuclear weapons , which Pyongyang claims is part of their right to self - defense .


fn_spacy_tokens_for_baseline: The raft capsized and Stephen drowned .

source_bert_toks: the raft caps ##ized and stephen drowned .

reference_raw: The raft's capsized, and Stephen's sunk.

reference_spacy_tokens: The raft 's capsized , and Stephen 's sunk .


fn_spacy_tokens_for_baseline: ( He 's underneath the table )

source_bert_toks: ( he ' s underneath the table )

reference_raw: (It's under the table)

reference_spacy_tokens: ( It 's under the table )


fn_spacy_tokens_for_baseline: It is one of those cases in which there is no neutrality to be had , because neutrality needs as much justification as any other position .

source_bert_toks: it is one of those cases in which there is no neutrality to be had , because neutrality needs as much justification as any other position .

reference_raw: This is one of those situations where neutrality is not necessary, since neutrality must be justified as well as any other situation.

reference_spacy_tokens: This is one of those situations where neutrality is not necessary , since neutrality must be justified as well as any other situation .


fn_spacy_tokens_for_baseline: In January - February 1997 , China supplied Iran with 40,000 barrels of calcium hypochlorite , a chemical - biological - radiological decontamination agent .

source_bert_toks: in january - february 1997 , china supplied iran with 40 , 000 barrels of calcium h ##yp ##och ##lor ##ite , a chemical - biological - radio ##logical deco ##nta ##mina ##tion agent .

reference_raw: In January-February 1997, China supplied Iran with 40,000 barrels of calcium hypochlorite, a chemical-biological-radiological decontamination substance.

reference_spacy_tokens: In January - February 1997 , China supplied Iran with 40,000 barrels of calcium hypochlorite , a chemical - biological - radiological decontamination substance .


fn_spacy_tokens_for_baseline: A flock of dunlin flew across the marsh in a silver swirl , catching the sun , dazzling the eye .

source_bert_toks: a flock of dun ##lin flew across the marsh in a silver swirl , catching the sun , dazzling the eye .

reference_raw: A herd of dunlin flew across the swamp in a silver swirl, catching the sun and blinding the eye.

reference_spacy_tokens: A herd of dunlin flew across the swamp in a silver swirl , catching the sun and blinding the eye .


fn_spacy_tokens_for_baseline: It 's got to me how these guys were looking at me .

source_bert_toks: it ' s got to me how these guys were looking at me .

reference_raw: I can see the way these people looked at me.

reference_spacy_tokens: I can see the way these people looked at me .


fn_spacy_tokens_for_baseline: The contempt lay in the fact that the issues would have been prejudged and improper public pressure would have been exerted on Distillers to cause them to settle .

source_bert_toks: the contempt lay in the fact that the issues would have been pre ##ju ##dge ##d and improper public pressure would have been ex ##erted on di ##sti ##ller ##s to cause them to settle .

reference_raw: The contempt was that the problems would be prejudged, and that the distillers would be under unfair public pressure to bring them to an end.

reference_spacy_tokens: The contempt was that the problems would be prejudged , and that the distillers would be under unfair public pressure to bring them to an end .


fn_spacy_tokens_for_baseline: Roman weapons used the bow as their mechanical basis , but the medieval ones tended to use the sling , which made them heavier and more cumbersome .

source_bert_toks: roman weapons used the bow as their mechanical basis , but the medieval ones tended to use the sling , which made them heavier and more cum ##bers ##ome .

reference_raw: The Roman weapons used the crossbow as a mechanical basis, but the medieval ones tended to use the sling, making it heavier and cumbersome.

reference_spacy_tokens: The Roman weapons used the crossbow as a mechanical basis , but the medieval ones tended to use the sling , making it heavier and cumbersome .


fn_spacy_tokens_for_baseline: The Arab world 's reaction to Barak 's victory was generally suspicious .

source_bert_toks: the arab world ' s reaction to barak ' s victory was generally suspicious .

reference_raw: The reaction of the Arab world to Barak's victory was generally suspect.

reference_spacy_tokens: The reaction of the Arab world to Barak 's victory was generally suspect .


fn_spacy_tokens_for_baseline: It was also a source of fees for more distant associates and although such relationships were more vulnerable to dynastic change , because less bound up with the territorial dominance of the lord of Middleham , some did nevertheless survive the transfer of power in 1471 .

source_bert_toks: it was also a source of fees for more distant associates and although such relationships were more vulnerable to d ##yna ##stic change , because less bound up with the territorial dominance of the lord of middle ##ham , some did nevertheless survive the transfer of power in 147 ##1 .

reference_raw: It was also a source of fees for more distant associates, and while such relationships were more vulnerable to dynamic change, because they were less bound by the Lord of Middleham's territorial dominance, some of them still lived through the transfer of power in 1471.

reference_spacy_tokens: It was also a source of fees for more distant associates , and while such relationships were more vulnerable to dynamic change , because they were less bound by the Lord of Middleham 's territorial dominance , some of them still lived through the transfer of power in 1471 .


fn_spacy_tokens_for_baseline: The animals show a sociable disposition that contrasts markedly with the isolationist features of the orang - utans , the ' loner ' among apes .

source_bert_toks: the animals show a soc ##iable disposition that contrasts markedly with the isolation ##ist features of the or ##ang - uta ##ns , the ' lone ##r ' among apes .

reference_raw: Animals exhibit a social disposition which contrasts significantly with the isolationist features of the orang-utans, the "loner" of the apes.

reference_spacy_tokens: Animals exhibit a social disposition which contrasts significantly with the isolationist features of the orang - utans , the " loner " of the apes .


fn_spacy_tokens_for_baseline: When they commissioned Richard Rogers as their architect , the wheel had come almost full circle .

source_bert_toks: when they commissioned richard rogers as their architect , the wheel had come almost full circle .

reference_raw: When Richard Rogers was assigned as their architect, the wheel was almost complete.

reference_spacy_tokens: When Richard Rogers was assigned as their architect , the wheel was almost complete .


fn_spacy_tokens_for_baseline: And his lean , strong body with its broad shoulders , tapering waist , narrow hips and long muscular legs exuded a virile , potent energy that seemed to crackle almost audibly .

source_bert_toks: and his lean , strong body with its broad shoulders , tape ##ring waist , narrow hips and long muscular legs ex ##uded a vi ##ril ##e , potent energy that seemed to crack ##le almost audi ##bly .

reference_raw: And his slender, strong body, broad-shouldered, tapering waist, narrow hips, and long-muscled legs, emitted a virile, powerful energy that seemed to crackle almost loudly.

reference_spacy_tokens: And his slender , strong body , broad - shouldered , tapering waist , narrow hips , and long - muscled legs , emitted a virile , powerful energy that seemed to crackle almost loudly .


fn_spacy_tokens_for_baseline: I had been invited to a banquet by some mad Russian prince .

source_bert_toks: i had been invited to a banquet by some mad russian prince .

reference_raw: Some crazy Russian prince invited me to a feast.

reference_spacy_tokens: Some crazy Russian prince invited me to a feast .


fn_spacy_tokens_for_baseline: And , as well as that final report from Alison , our weekly phone - in tomorrow will also be on the subject of child care .

source_bert_toks: and , as well as that final report from alison , our weekly phone - in tomorrow will also be on the subject of child care .

reference_raw: And in addition to Alison's final report, our week-long call tomorrow will also be about childcare.

reference_spacy_tokens: And in addition to Alison 's final report , our week - long call tomorrow will also be about childcare .


fn_spacy_tokens_for_baseline: As he was doing that , he saw the paragraph about himself in The Stage and planted his shoe over it .

source_bert_toks: as he was doing that , he saw the paragraph about himself in the stage and planted his shoe over it .

reference_raw: As he did so, he saw a paragraph about himself in the Stage and put his boot on it.

reference_spacy_tokens: As he did so , he saw a paragraph about himself in the Stage and put his boot on it .


fn_spacy_tokens_for_baseline: It has been thrown away in the haste of cobbling together a package of ' tough new laws ' whenever the Government felt the need for legislative machismo .

source_bert_toks: it has been thrown away in the haste of cobb ##ling together a package of ' tough new laws ' whenever the government felt the need for legislative mach ##ism ##o .

reference_raw: When the government felt the need for legislative machism, it was thrown away in a hurry to cobble together a set of 'hard new laws'.

reference_spacy_tokens: When the government felt the need for legislative machism , it was thrown away in a hurry to cobble together a set of ' hard new laws ' .


fn_spacy_tokens_for_baseline: Trying to do things on the cheap only results in an inferior service .

source_bert_toks: trying to do things on the cheap only results in an inferior service .

reference_raw: Trying to do something cheap only leads to a lower service.

reference_spacy_tokens: Trying to do something cheap only leads to a lower service .


fn_spacy_tokens_for_baseline: Commonly known as an essay , this form of assessment is the most open - ended type of written test .

source_bert_toks: commonly known as an essay , this form of assessment is the most open - ended type of written test .

reference_raw: Commonly known as an essay, this form of assessment is the most open-ended type of written examination.

reference_spacy_tokens: Commonly known as an essay , this form of assessment is the most open - ended type of written examination .


fn_spacy_tokens_for_baseline: She has written to the Attorney General protesting that the courts have been too lenient on Dr Courtney and she has tabled a Parliamentary Question to Sir Nicholas designed to force a judicial review of the sentence .

source_bert_toks: she has written to the attorney general protesting that the courts have been too len ##ient on dr courtney and she has table ##d a parliamentary question to sir nicholas designed to force a judicial review of the sentence .

reference_raw: She wrote to the Attorney General protesting that Dr. Courtney was too kind to the court, and put a parliamentary question to Sir Nicholas to force a judicial review of the sentence.

reference_spacy_tokens: She wrote to the Attorney General protesting that Dr. Courtney was too kind to the court , and put a parliamentary question to Sir Nicholas to force a judicial review of the sentence .


fn_spacy_tokens_for_baseline: The organisers , with pledges of support from numerous internationals , are attempting an intriguing second - row pairing of Matt Keenan ( Western Samoa ) and Sam Domoni ( Fiji ) .

source_bert_toks: the organise ##rs , with pledge ##s of support from numerous internationals , are attempting an intriguing second - row pairing of matt keenan ( western samoa ) and sam dom ##oni ( fiji ) .

reference_raw: The organizers, with promises of support from a number of internationals, are trying to find Matt Keenan (Western Samoa) and Sam Domoni (Fiji).

reference_spacy_tokens: The organizers , with promises of support from a number of internationals , are trying to find Matt Keenan ( Western Samoa ) and Sam Domoni ( Fiji ) .


fn_spacy_tokens_for_baseline: New Zealand notes that because many of its young professionals are disenchanted and disillusioned , there is a gradual disappearance of ' traditional choirs ' .

source_bert_toks: new zealand notes that because many of its young professionals are di ##sen ##chan ##ted and di ##sil ##lusion ##ed , there is a gradual disappearance of ' traditional choirs ' .

reference_raw: New Zealand notes that because many of its young professionals are dissatisfied and disillusioned, "traditional choirs" are gradually disappearing.

reference_spacy_tokens: New Zealand notes that because many of its young professionals are dissatisfied and disillusioned , " traditional choirs " are gradually disappearing .


fn_spacy_tokens_for_baseline: Try to gather a diversity of views , not just one approach to the question .

source_bert_toks: try to gather a diversity of views , not just one approach to the question .

reference_raw: Try to collect different views, not just one approach to the question.

reference_spacy_tokens: Try to collect different views , not just one approach to the question .


fn_spacy_tokens_for_baseline: In fact , there 's no end to the things that irritate me about television news -- the inane scripts , for one .

source_bert_toks: in fact , there ' s no end to the things that ir ##rita ##te me about television news - - the ina ##ne scripts , for one .

reference_raw: In fact, there is no end to the things that irritate me about the TV news, the inadvertent scripts, for one.

reference_spacy_tokens: In fact , there is no end to the things that irritate me about the TV news , the inadvertent scripts , for one .


fn_spacy_tokens_for_baseline: The latest Bulletin of the American Mathematical Society resolves a question that came to light in the days of Lord Rayleigh , who showed how an idealised object called a black body radiates heat and light energy .

source_bert_toks: the latest bulletin of the american mathematical society resolve ##s a question that came to light in the days of lord ray ##leigh , who showed how an ideal ##ised object called a black body ra ##dia ##tes heat and light energy .

reference_raw: The latest bulletin of the American Mathematics Society solves a question that came to light in the days of Lord Rayleigh, who has shown how an idealized object called a black body emits heat and light energy.

reference_spacy_tokens: The latest bulletin of the American Mathematics Society solves a question that came to light in the days of Lord Rayleigh , who has shown how an idealized object called a black body emits heat and light energy .


fn_spacy_tokens_for_baseline: As with all advice to clients , written notification must follow the spoken word .

source_bert_toks: as with all advice to clients , written notification must follow the spoken word .

reference_raw: As with all advice to clients, written notices must follow the spoken word.

reference_spacy_tokens: As with all advice to clients , written notices must follow the spoken word .


fn_spacy_tokens_for_baseline: This can be positioned in the rear of the car , but never be tempted to put a dog in a cage in the boot ( trunk ) of a car .

source_bert_toks: this can be positioned in the rear of the car , but never be tempted to put a dog in a cage in the boot ( trunk ) of a car .

reference_raw: You can put it in the back of the car, but you can never be tempted to put a dog in a cage in the trunk of the car.

reference_spacy_tokens: You can put it in the back of the car , but you can never be tempted to put a dog in a cage in the trunk of the car .


fn_spacy_tokens_for_baseline: A YOUNG policeman helped to crack a lorry hijacking , the Crown claimed yesterday , because he had been in school with the man accused of abducting the driver and stealing spirits , food and tobacco .

source_bert_toks: a young policeman helped to crack a lo ##rry hi ##jack ##ing , the crown claimed yesterday , because he had been in school with the man accused of abd ##uc ##ting the driver and stealing spirits , food and tobacco .

reference_raw: A young cop helped solve a truck robbery, the Crown said yesterday, because he was in school with a man accused of kidnapping a driver and stealing liquor, food, and tobacco.

reference_spacy_tokens: A young cop helped solve a truck robbery , the Crown said yesterday , because he was in school with a man accused of kidnapping a driver and stealing liquor , food , and tobacco .


fn_spacy_tokens_for_baseline: The warmth of the ship and want of air had occasioned the skunk - weed to put forth two fine blossoms , very beautiful . '

source_bert_toks: the warmth of the ship and want of air had occasion ##ed the sk ##unk - weed to put forth two fine blossoms , very beautiful . '

reference_raw: The warmth of the ship and the need for air had given the skunk-weeds the opportunity to produce two fine flowers, very beautiful.'

reference_spacy_tokens: The warmth of the ship and the need for air had given the skunk - weeds the opportunity to produce two fine flowers , very beautiful . '


fn_spacy_tokens_for_baseline: Guangzhou is famous for its food .

source_bert_toks: guangzhou is famous for its food .

reference_raw: Guangzhou is well-known for his food.

reference_spacy_tokens: Guangzhou is well - known for his food .


fn_spacy_tokens_for_baseline: She was a very small woman , and she was often tired because of the children .

source_bert_toks: she was a very small woman , and she was often tired because of the children .

reference_raw: She was a very small woman, and the children made her often weary.

reference_spacy_tokens: She was a very small woman , and the children made her often weary .


fn_spacy_tokens_for_baseline: My appearance , which Helmut had embellished with jewellery , expensive clothes and regular visits to beauty salons , Jean - Claude liked unadorned and without make - up .

source_bert_toks: my appearance , which helmut had em ##bell ##ished with jewellery , expensive clothes and regular visits to beauty salon ##s , jean - claude liked una ##dor ##ned and without make - up .

reference_raw: Jean-Claude liked my appearance, which Helmut had embossed in jewellery, expensive clothing, and regular visits to the beauty parlors.

reference_spacy_tokens: Jean - Claude liked my appearance , which Helmut had embossed in jewellery , expensive clothing , and regular visits to the beauty parlors .


fn_spacy_tokens_for_baseline: ( b ) the first rise ( after 300 yds ) is High Cliff which is the highest cliff in Cornwall .

source_bert_toks: ( b ) the first rise ( after 300 yd ##s ) is high cliff which is the highest cliff in cornwall .

reference_raw: (b) the first rise (after 300 yds) is the highest reef in Cornwall.

reference_spacy_tokens: ( b ) the first rise ( after 300 yds ) is the highest reef in Cornwall .


fn_spacy_tokens_for_baseline: Therese folded her tea - towel and hung it on the silver bar in front of the range .

source_bert_toks: therese folded her tea - towel and hung it on the silver bar in front of the range .

reference_raw: Theresa unfolded her tea-towel and hung it on a silver bar in front of the range.

reference_spacy_tokens: Theresa unfolded her tea - towel and hung it on a silver bar in front of the range .


fn_spacy_tokens_for_baseline: Even the classical certainties of laboratory apparatus present us with a mystery when we recognise that we do not fully comprehend how they arise from the quantum substrate of which they are composed .

source_bert_toks: even the classical certain ##ties of laboratory apparatus present us with a mystery when we recognise that we do not fully comprehend how they arise from the quantum substrate of which they are composed .

reference_raw: Even the classic certainty of the laboratory apparatus presents a mystery to us when we recognize that we do not fully understand how it stems from the quantum substrate of which it is composed.

reference_spacy_tokens: Even the classic certainty of the laboratory apparatus presents a mystery to us when we recognize that we do not fully understand how it stems from the quantum substrate of which it is composed .


fn_spacy_tokens_for_baseline: Miss Jenny ought to be more thoughtful instead of causing extra work . '

source_bert_toks: miss jenny ought to be more thoughtful instead of causing extra work . '

reference_raw: Miss Jenny should be more conscientious than giving extra work.'

reference_spacy_tokens: Miss Jenny should be more conscientious than giving extra work . '


fn_spacy_tokens_for_baseline: Publicity officer Maurice Catchpole said the project , which involved co - operation between rotary clubs in England and Holland , was designed to rekindle the much - needed aid programme .

source_bert_toks: publicity officer maurice catch ##pole said the project , which involved co - operation between rotary clubs in england and holland , was designed to re ##kind ##le the much - needed aid programme .

reference_raw: Publicity official Maurice Catchpole stated that the project, which included cooperation between rotary clubs in England and Holland, was designed to restore a much-needed support programme.

reference_spacy_tokens: Publicity official Maurice Catchpole stated that the project , which included cooperation between rotary clubs in England and Holland , was designed to restore a much - needed support programme .


fn_spacy_tokens_for_baseline: This story is really made up of two stories : A How Rima caught and caged the cassowary B How the Bower - bird tricked the Cassowary .

source_bert_toks: this story is really made up of two stories : a how rim ##a caught and cage ##d the cass ##owa ##ry b how the bow ##er - bird tricked the cass ##owa ##ry .

reference_raw: The story is really made up of two stories: the way Rima caught the cassowary, the way the Bower bird tricked the Cassowary.

reference_spacy_tokens: The story is really made up of two stories : the way Rima caught the cassowary , the way the Bower bird tricked the Cassowary .


fn_spacy_tokens_for_baseline: In practice the actions of governments have been less high - minded : a fact the opposition is always quick to point out .

source_bert_toks: in practice the actions of governments have been less high - minded : a fact the opposition is always quick to point out .

reference_raw: In practice, government action is less subtle: the opposition is always quick to point out.

reference_spacy_tokens: In practice , government action is less subtle : the opposition is always quick to point out .


fn_spacy_tokens_for_baseline: Tom 's work mostly involves checking for sharp teeth and rasping them smooth .

source_bert_toks: tom ' s work mostly involves checking for sharp teeth and ras ##ping them smooth .

reference_raw: Tom's work consists mostly of checking for teeth that are razor-sharp and rasping them smoothly.

reference_spacy_tokens: Tom 's work consists mostly of checking for teeth that are razor - sharp and rasping them smoothly .


fn_spacy_tokens_for_baseline: Sheikh Mohammed , who is also the UAE Defence Minister , said during the launching ceremony " we wish to make Dubai a center of modern economy " .

source_bert_toks: sheikh mohammed , who is also the uae defence minister , said during the launching ceremony " we wish to make dubai a center of modern economy " .

reference_raw: Sheik Mohammed, who is also the Secretary of Defense of the United Arab Emirates, said during the opening ceremony, "We want to make Dubai the centre of the modern economy."

reference_spacy_tokens: Sheik Mohammed , who is also the Secretary of Defense of the United Arab Emirates , said during the opening ceremony , " We want to make Dubai the centre of the modern economy . "


fn_spacy_tokens_for_baseline: There are 12 of them ; you ca n't miss the eye - popping Lisboa , which has several floors of gambling , but there 's also the Hyatt Regency , the Mandarian Oriental , and Taipa 's Jockey Club .

source_bert_toks: there are 12 of them ; you can ' t miss the eye - popping li ##sb ##oa , which has several floors of gambling , but there ' s also the h ##yat ##t regency , the man ##dar ##ian oriental , and tai ##pa ' s jockey club .

reference_raw: There are twelve of them; you can't miss Lisboa, who's got a few floors of gambling, but there's also the Hyatt Regency, the Mandarian Oriental, and the Taipa Jockey Club.

reference_spacy_tokens: There are twelve of them ; you ca n't miss Lisboa , who 's got a few floors of gambling , but there 's also the Hyatt Regency , the Mandarian Oriental , and the Taipa Jockey Club .


fn_spacy_tokens_for_baseline: Presumably killers did not take kindly to amateur detectives .

source_bert_toks: presumably killers did not take kindly to amateur detectives .

reference_raw: Probably the killers didn't take kindly to amate detectives.

reference_spacy_tokens: Probably the killers did n't take kindly to amate detectives .


fn_spacy_tokens_for_baseline: Alongside the hotel runs busy Nathan Road , Hong Kong 's fabled shopping street , lined with shops , hotels , and restaurants .

source_bert_toks: alongside the hotel runs busy nathan road , hong kong ' s fable ##d shopping street , lined with shops , hotels , and restaurants .

reference_raw: Alongside the hotel is a busy Nathan Road, a legendary Hong Kong mall lined with stores, hotels, and restaurants.

reference_spacy_tokens: Alongside the hotel is a busy Nathan Road , a legendary Hong Kong mall lined with stores , hotels , and restaurants .


fn_spacy_tokens_for_baseline: " We have had cricket sales before and we enjoy seeing the buyers in front of the rostrum unscrewing their flasks and unwrapping their sandwiches , " said Richard Madley , a director of Phillips auctioneers .

source_bert_toks: " we have had cricket sales before and we enjoy seeing the buyers in front of the ro ##st ##rum un ##sc ##rew ##ing their fl ##ask ##s and un ##wr ##app ##ing their sandwiches , " said richard mad ##ley , a director of phillips auction ##eers .

reference_raw: "We've had cricket sales before, and we like to see the buyers outside the rostrum unpack their bottles and unpack their sandwiches," said Richard Madley, the director of the Phillips Auctioneers.

reference_spacy_tokens: " We 've had cricket sales before , and we like to see the buyers outside the rostrum unpack their bottles and unpack their sandwiches , " said Richard Madley , the director of the Phillips Auctioneers .


fn_spacy_tokens_for_baseline: A similar fate befell Adam Watson , who subsequently called for the quizmaster 's resignation .

source_bert_toks: a similar fate be ##fell adam watson , who subsequently called for the quiz ##master ' s resignation .

reference_raw: Adam Watson, who then called for the quizmaster's resignation, was struck by a similar fate.

reference_spacy_tokens: Adam Watson , who then called for the quizmaster 's resignation , was struck by a similar fate .


fn_spacy_tokens_for_baseline: The summer promised to be delicious , but nothing is ever perfect .

source_bert_toks: the summer promised to be delicious , but nothing is ever perfect .

reference_raw: Summer said it would be delicious, but nothing's ever perfect.

reference_spacy_tokens: Summer said it would be delicious , but nothing 's ever perfect .


fn_spacy_tokens_for_baseline: The local bishop , Johannes Hempel , spoke in all four churches , and called on the state to open a dialogue with the protesters .

source_bert_toks: the local bishop , johannes hem ##pel , spoke in all four churches , and called on the state to open a dialogue with the protesters .

reference_raw: The local bishop, Johannes Hempel, spoke in all four churches and called on the State to open a debate with the Protestants.

reference_spacy_tokens: The local bishop , Johannes Hempel , spoke in all four churches and called on the State to open a debate with the Protestants .


fn_spacy_tokens_for_baseline: Her sun was the biggest poacher -- he was a devil : he 'd rob your house in the middle of the day and let you see him !

source_bert_toks: her sun was the biggest po ##ache ##r - - he was a devil : he ' d rob your house in the middle of the day and let you see him !

reference_raw: Her sun was the greatest poacher - he was the devil: he would steal your house in the middle of the day and show it to you!

reference_spacy_tokens: Her sun was the greatest poacher - he was the devil : he would steal your house in the middle of the day and show it to you !


fn_spacy_tokens_for_baseline: It is possible , however , that this subject was discussed , since General Toufanian apparently attended a test - launch of a Jericho missile .

source_bert_toks: it is possible , however , that this subject was discussed , since general to ##uf ##anian apparently attended a test - launch of a jericho missile .

reference_raw: It is possible, however, that the subject was discussed, since General Toufanian appeared to have taken part in the test of the Jericho missile.

reference_spacy_tokens: It is possible , however , that the subject was discussed , since General Toufanian appeared to have taken part in the test of the Jericho missile .


fn_spacy_tokens_for_baseline: A GIFTED young lawyer is lured to a luxurious legal practice that serves the Mafia .

source_bert_toks: a gifted young lawyer is lured to a luxurious legal practice that serves the mafia .

reference_raw: The GIFTED Youth Lawyer is lured to a luxurious law firm that serves the Mafia.

reference_spacy_tokens: The GIFTED Youth Lawyer is lured to a luxurious law firm that serves the Mafia .


fn_spacy_tokens_for_baseline: You can school him over hurdles . '

source_bert_toks: you can school him over hurdles . '

reference_raw: You can teach him about obstacles.'

reference_spacy_tokens: You can teach him about obstacles . '


fn_spacy_tokens_for_baseline: Riot police using tear gas and water cannons dislodged several dozen self - proclaimed ' anarchists ' who had set fire to administrative offices at Athens Polytechnic University on the night of Oct. 24 - 25 .

source_bert_toks: riot police using tear gas and water cannons di ##sl ##od ##ged several dozen self - proclaimed ' anarchist ##s ' who had set fire to administrative offices at athens polytechnic university on the night of oct . 24 - 25 .

reference_raw: The riot police, using tear gas and water cannons, dislodged several dozen self-proclaimed anarchists who burned down the offices of the Athens Polytechnic University on the night of October 24-25.

reference_spacy_tokens: The riot police , using tear gas and water cannons , dislodged several dozen self - proclaimed anarchists who burned down the offices of the Athens Polytechnic University on the night of October 24 - 25 .


fn_spacy_tokens_for_baseline: Aggravations are minimised , or can be regulated .

source_bert_toks: ag ##gra ##vation ##s are mini ##mise ##d , or can be regulated .

reference_raw: Aggravation is minimised or may be regulated.

reference_spacy_tokens: Aggravation is minimised or may be regulated .


fn_spacy_tokens_for_baseline: Deputy Prime Minister and Foreign Minister Michio Watanabe proposed a new compromise solution on April 21 in an attempt to settle with Russia the long - standing territorial dispute over the four islands known by Russia as the southern Kuriles , and by Japan as the Northern Territories , situated to the north - east of Japan 's northern island of Hokkaido .

source_bert_toks: deputy prime minister and foreign minister mic ##hi ##o wat ##ana ##be proposed a new compromise solution on april 21 in an attempt to settle with russia the long - standing territorial dispute over the four islands known by russia as the southern ku ##ril ##es , and by japan as the northern territories , situated to the north - east of japan ' s northern island of hokkaido .

reference_raw: On April 21, Deputy Prime Minister and Foreign Minister Michio Watanabe proposed a new compromise solution in order to resolve the long-term territorial dispute between Russia and the four islands known as the southern Kuriles and Japan as the northern territories located in the north-east of Japan's northern island of Hokkaido.

reference_spacy_tokens: On April 21 , Deputy Prime Minister and Foreign Minister Michio Watanabe proposed a new compromise solution in order to resolve the long - term territorial dispute between Russia and the four islands known as the southern Kuriles and Japan as the northern territories located in the north - east of Japan 's northern island of Hokkaido .


fn_spacy_tokens_for_baseline: She thought of Yevdoxia , his wife , dragging his decayed body and bruised mind around .

source_bert_toks: she thought of ye ##vd ##ox ##ia , his wife , dragging his decay ##ed body and bruised mind around .

reference_raw: She thought of Yevdoxia, his wife, dragging his decomposed body and his bruised mind.

reference_spacy_tokens: She thought of Yevdoxia , his wife , dragging his decomposed body and his bruised mind .


fn_spacy_tokens_for_baseline: Launched on a miniscule budget and a surge of dubious optimism , held together in the first eighteen months by a poultice of desperation and blind faith ( not to mention liberal applications of cow gum and sellotape ) , still independent after a decade of brinkmanship publishing this is a tremendous achievement .

source_bert_toks: launched on a mini ##scu ##le budget and a surge of dubious optimism , held together in the first eighteen months by a po ##ult ##ice of desperation and blind faith ( not to mention liberal applications of cow gum and sell ##ota ##pe ) , still independent after a decade of brink ##manship publishing this is a tremendous achievement .

reference_raw: Starting with a miniscule budget and a surge of dubious optimism, held together for the first eighteen months by a crowd of despair and blind faith (not to mention liberal uses of cow gum and sellotape), which is still independent after ten years of cutting-edge publication, is a huge success.

reference_spacy_tokens: Starting with a miniscule budget and a surge of dubious optimism , held together for the first eighteen months by a crowd of despair and blind faith ( not to mention liberal uses of cow gum and sellotape ) , which is still independent after ten years of cutting - edge publication , is a huge success .


fn_spacy_tokens_for_baseline: The others also draped their underclothes among the branches .

source_bert_toks: the others also draped their under ##cloth ##es among the branches .

reference_raw: The others also draped their underwear among the branches.

reference_spacy_tokens: The others also draped their underwear among the branches .


fn_spacy_tokens_for_baseline: Place names are often based on the distance to the nearest mile - post , as measured from the tip of the Kowloon peninsula .

source_bert_toks: place names are often based on the distance to the nearest mile - post , as measured from the tip of the ko ##wl ##oon peninsula .

reference_raw: Place names are frequently based on the distance from the nearest mile point measured from the tip of the Kowloon peninsula.

reference_spacy_tokens: Place names are frequently based on the distance from the nearest mile point measured from the tip of the Kowloon peninsula .


fn_spacy_tokens_for_baseline: They straggled lamentably along the road , bent in two by the weight of their packs , streaming with water , and all this to go and become mashed to pulp in muddy trenches .

source_bert_toks: they st ##rag ##gled lame ##nta ##bly along the road , bent in two by the weight of their packs , streaming with water , and all this to go and become mas ##hed to pulp in muddy trenches .

reference_raw: They were limping along the road, bending in two by the weight of their packs, streaming with water, and all that to go and be mashed to a pulp in the muddy trenches.

reference_spacy_tokens: They were limping along the road , bending in two by the weight of their packs , streaming with water , and all that to go and be mashed to a pulp in the muddy trenches .


fn_spacy_tokens_for_baseline: This makes it unobtrusive and less prone to damage .

source_bert_toks: this makes it uno ##bt ##rus ##ive and less prone to damage .

reference_raw: That makes it inconspicuous and less vulnerable to damage.

reference_spacy_tokens: That makes it inconspicuous and less vulnerable to damage .


fn_spacy_tokens_for_baseline: 2 ) Congressional representatives have two basic responsibilities while voting in office -- dealing with national issues ( programmatic actions such as casting roll call votes on legislation that imposes costs and/or confers benefits on the population at large ) and attending to local issues ( constituency service and pork barrel ) .

source_bert_toks: 2 ) congressional representatives have two basic responsibilities while voting in office - - dealing with national issues ( program ##matic actions such as casting roll call votes on legislation that impose ##s costs and / or con ##fers benefits on the population at large ) and attending to local issues ( constituency service and pork barrel ) .

reference_raw: 2) Congressional representatives, when they vote in office, have two basic responsibilities: to address domestic issues (programmatic measures such as the casting of roll-call votes on legislation that imposes costs and/or benefits for the general population) and to address local issues (constituency services and pork barrels).

reference_spacy_tokens: 2 ) Congressional representatives , when they vote in office , have two basic responsibilities : to address domestic issues ( programmatic measures such as the casting of roll - call votes on legislation that imposes costs and/or benefits for the general population ) and to address local issues ( constituency services and pork barrels ) .


fn_spacy_tokens_for_baseline: Afghan authorities say the insurgents are seeking to tap into the outrage generated by the church 's threat to whip up fury against Western forces and President Hamid Karzai .

source_bert_toks: afghan authorities say the insurgents are seeking to tap into the outrage generated by the church ' s threat to whip up fury against western forces and president hamid ka ##rza ##i .

reference_raw: According to the Afghan authorities, the rebels are trying to tap into the indignation caused by the Church's threat of whipping up violence against Western forces and President Hamid Karzai.

reference_spacy_tokens: According to the Afghan authorities , the rebels are trying to tap into the indignation caused by the Church 's threat of whipping up violence against Western forces and President Hamid Karzai .


fn_spacy_tokens_for_baseline: Rosemary Sandford of Darlington Association on Disability , said they were saddened and disappointed they were not given the opportunity to express their views at the meeting .

source_bert_toks: rosemary sand ##ford of darlington association on disability , said they were sad ##dened and disappointed they were not given the opportunity to express their views at the meeting .

reference_raw: Rosemary Sandford of Darlington Association for Disability said that they were sad and disappointed that they had not been given the opportunity to comment at the meeting.

reference_spacy_tokens: Rosemary Sandford of Darlington Association for Disability said that they were sad and disappointed that they had not been given the opportunity to comment at the meeting .


fn_spacy_tokens_for_baseline: Continuing around the coast in a counter - clockwise direction , Deep Water Bay offers a good beach and harbors .

source_bert_toks: continuing around the coast in a counter - clockwise direction , deep water bay offers a good beach and harbor ##s .

reference_raw: Deep Water Bay, which continues to circle the coast in a counterclockwise direction, provides a good beach and port.

reference_spacy_tokens: Deep Water Bay , which continues to circle the coast in a counterclockwise direction , provides a good beach and port .


fn_spacy_tokens_for_baseline: It was an eerie feeling to be stealing through the forest like this .

source_bert_toks: it was an eerie feeling to be stealing through the forest like this .

reference_raw: It was a strange feeling to sneak through the woods like that.

reference_spacy_tokens: It was a strange feeling to sneak through the woods like that .


fn_spacy_tokens_for_baseline: He did n't support our strikes and we used to row about it , but not now .

source_bert_toks: he didn ' t support our strikes and we used to row about it , but not now .

reference_raw: He didn't support our strikes, and we fought around, but not now.

reference_spacy_tokens: He did n't support our strikes , and we fought around , but not now .


fn_spacy_tokens_for_baseline: ' She 'd have been better with someone nearer her own age , ' the old woman murmured to herself .

source_bert_toks: ' she ' d have been better with someone nearer her own age , ' the old woman murmured to herself .

reference_raw: "She'd be better off with someone closer to her age," the old woman whispered to herself.

reference_spacy_tokens: " She 'd be better off with someone closer to her age , " the old woman whispered to herself .


fn_spacy_tokens_for_baseline: Private benefactors enabled the school to build two new dormitories , while thirty of the youngest children were accommodated in an old farmhouse .

source_bert_toks: private benefactor ##s enabled the school to build two new dorm ##itor ##ies , while thirty of the youngest children were accommodated in an old farmhouse .

reference_raw: Private benefactors allowed the school to build two new bedrooms, and thirty of the youngest children were housed in an old farmhouse.

reference_spacy_tokens: Private benefactors allowed the school to build two new bedrooms , and thirty of the youngest children were housed in an old farmhouse .


fn_spacy_tokens_for_baseline: But still he says he would not be married or sober without the incident .

source_bert_toks: but still he says he would not be married or sober without the incident .

reference_raw: But he's still saying that without the incident, he wouldn't have been married or sobriety.

reference_spacy_tokens: But he 's still saying that without the incident , he would n't have been married or sobriety .


fn_spacy_tokens_for_baseline: A crowd that seemed trapped within the boundaries of the market trudged the muddy aisles and picked at merchandise .

source_bert_toks: a crowd that seemed trapped within the boundaries of the market tr ##udged the muddy aisles and picked at merchandise .

reference_raw: A crowd that seemed to be trapped within the edges of the market plodded through the muddy alleys, picking up the goods.

reference_spacy_tokens: A crowd that seemed to be trapped within the edges of the market plodded through the muddy alleys , picking up the goods .


fn_spacy_tokens_for_baseline: ROBIN Smith , Hampshire 's defiant middle - order maestro , yesterday had the last word in his summer - long bitter duel with Waqar Younis .

source_bert_toks: robin smith , hampshire ' s defiant middle - order maestro , yesterday had the last word in his summer - long bitter duel with wa ##qa ##r you ##nis .

reference_raw: ROBIN Smith, Hampshire's defiant middle-order master, had the last word of his summer bitter fight with Waqar Younis.

reference_spacy_tokens: ROBIN Smith , Hampshire 's defiant middle - order master , had the last word of his summer bitter fight with Waqar Younis .


fn_spacy_tokens_for_baseline: One of these walls , the Western Wall , is today a major reminder of Jerusalem 's greatness under Herod .

source_bert_toks: one of these walls , the western wall , is today a major reminder of jerusalem ' s great ##ness under hero ##d .

reference_raw: One of these walls, the western wall, is now an important reminder of the greatness of Jerusalem under Herod.

reference_spacy_tokens: One of these walls , the western wall , is now an important reminder of the greatness of Jerusalem under Herod .


fn_spacy_tokens_for_baseline: Zambia hunched miserably on a plump floor - cushion , awaiting execution of hir sentence .

source_bert_toks: zambia hunched mis ##era ##bly on a plump floor - cushion , awaiting execution of hi ##r sentence .

reference_raw: Zambia crouched unhappily on the plump floor and waited for his sentence to be executed.

reference_spacy_tokens: Zambia crouched unhappily on the plump floor and waited for his sentence to be executed .


fn_spacy_tokens_for_baseline: This , they felt , was symptomatic of the paper 's commendable interest in and support of , the arts .

source_bert_toks: this , they felt , was sy ##mpt ##oma ##tic of the paper ' s com ##men ##dable interest in and support of , the arts .

reference_raw: They felt that this was an embodiment of the paper's admirable interest in and support for the arts.

reference_spacy_tokens: They felt that this was an embodiment of the paper 's admirable interest in and support for the arts .


fn_spacy_tokens_for_baseline: The Isle of Man is in the middle of the Irish Sea , halfway there .

source_bert_toks: the isle of man is in the middle of the irish sea , halfway there .

reference_raw: The Isle of Man is halfway across the Irish Sea.

reference_spacy_tokens: The Isle of Man is halfway across the Irish Sea .


fn_spacy_tokens_for_baseline: Oh , Billie , I must have been crazy to let you come with me .

source_bert_toks: oh , billie , i must have been crazy to let you come with me .

reference_raw: Billie, I had to be a fool to let you go with me.

reference_spacy_tokens: Billie , I had to be a fool to let you go with me .


fn_spacy_tokens_for_baseline: Hughes ' actions would have beneficial repercussions , both immediate and lasting .

source_bert_toks: hughes ' actions would have beneficial rep ##er ##cus ##sions , both immediate and lasting .

reference_raw: Hughes' actions would have positive consequences, both immediate and permanent.

reference_spacy_tokens: Hughes ' actions would have positive consequences , both immediate and permanent .


fn_spacy_tokens_for_baseline: It seems its very high speeds are only achieved occasionally , when pursuing food or escaping , and it normally cruises at about a third or a half of its rapid dashes .

source_bert_toks: it seems its very high speeds are only achieved occasionally , when pursuing food or escaping , and it normally cruises at about a third or a half of its rapid dash ##es .

reference_raw: It appears that its very high speeds are achieved only occasionally in pursuit of food or escape, and that it usually travels at about a third or a half of its speed.

reference_spacy_tokens: It appears that its very high speeds are achieved only occasionally in pursuit of food or escape , and that it usually travels at about a third or a half of its speed .


fn_spacy_tokens_for_baseline: He took a step towards her and for one heart - stopping moment she thought that he was going to kiss her , and then where would she and her collection of high - minded resolutions be ?

source_bert_toks: he took a step towards her and for one heart - stopping moment she thought that he was going to kiss her , and then where would she and her collection of high - minded resolutions be ?

reference_raw: He took a step toward her, and for a heartbeat she thought he would kiss her, and then where would she and her collection of high-minded resolutions be?

reference_spacy_tokens: He took a step toward her , and for a heartbeat she thought he would kiss her , and then where would she and her collection of high - minded resolutions be ?


fn_spacy_tokens_for_baseline: A scheme to help the unemployed homeless find a job and accommodation , pioneered in London by The Industrial Society and major employers , is to be extended to Bristol , Edinburgh , Hull and Southend in Essex .

source_bert_toks: a scheme to help the unemployed homeless find a job and accommodation , pioneered in london by the industrial society and major employers , is to be extended to bristol , edinburgh , hull and southend in essex .

reference_raw: Bristol, Edinburgh, Hull and Southend in Essex should be extended to the scheme to help unemployed homeless people to find employment and housing, which is a front-runner in London by the Industrial Society and major employers.

reference_spacy_tokens: Bristol , Edinburgh , Hull and Southend in Essex should be extended to the scheme to help unemployed homeless people to find employment and housing , which is a front - runner in London by the Industrial Society and major employers .


fn_spacy_tokens_for_baseline: This is a remarkable result when you consider the statistical improbability of the same large molecule spontaneously arising twice .

source_bert_toks: this is a remarkable result when you consider the statistical imp ##ro ##ba ##bility of the same large molecule spontaneously arising twice .

reference_raw: This is a remarkable result if you consider statistically unlikely that the same large molecule will spontaneously occur twice.

reference_spacy_tokens: This is a remarkable result if you consider statistically unlikely that the same large molecule will spontaneously occur twice .


fn_spacy_tokens_for_baseline: ' Can you suggest any reason why anyone might have wanted to kill Matthew ? '

source_bert_toks: ' can you suggest any reason why anyone might have wanted to kill matthew ? '

reference_raw: "Can you think of any reason why anyone would want to kill Matthew?"

reference_spacy_tokens: " Can you think of any reason why anyone would want to kill Matthew ? "


fn_spacy_tokens_for_baseline: While I appreciate U.S. News and World Report 's endorsement , the true value of your support is measured by the way Goodwill takes on problems that affect all of us .

source_bert_toks: while i appreciate u . s . news and world report ' s endorsement , the true value of your support is measured by the way goodwill takes on problems that affect all of us .

reference_raw: While I appreciate the support of the United States News and the World Report, the true worth of your support is measured by the way Goodwill deals with the problems that affect us all.

reference_spacy_tokens: While I appreciate the support of the United States News and the World Report , the true worth of your support is measured by the way Goodwill deals with the problems that affect us all .


fn_spacy_tokens_for_baseline: We have encountered them throughout the book .

source_bert_toks: we have encountered them throughout the book .

reference_raw: We've met them all over the book.

reference_spacy_tokens: We 've met them all over the book .


fn_spacy_tokens_for_baseline: The shortest route , depending on weapon design , could be to purchase or steal fissile material .

source_bert_toks: the shortest route , depending on weapon design , could be to purchase or steal fis ##sil ##e material .

reference_raw: Depending on the construction of the weapon, the shortest route could be to buy or steal fissile material.

reference_spacy_tokens: Depending on the construction of the weapon , the shortest route could be to buy or steal fissile material .


fn_spacy_tokens_for_baseline: Gallagher brought his arm back and over and laid the lash across Luke 's bare back .

source_bert_toks: gallagher brought his arm back and over and laid the lash across luke ' s bare back .

reference_raw: Gallagher brought his hand back and forth and laid the whip on Luke's bare back.

reference_spacy_tokens: Gallagher brought his hand back and forth and laid the whip on Luke 's bare back .


fn_spacy_tokens_for_baseline: He kept meticulous notes of his drug habit , including the formula he used to obtain his ' target ' drug , MPPP , a synthetic heroin .

source_bert_toks: he kept met ##ic ##ulous notes of his drug habit , including the formula he used to obtain his ' target ' drug , mp ##pp , a synthetic heroin .

reference_raw: He kept careful notes of his drug habits, including the formula he used to get his "target" drug, MPPP, synthetic heroine.

reference_spacy_tokens: He kept careful notes of his drug habits , including the formula he used to get his " target " drug , MPPP , synthetic heroine .


fn_spacy_tokens_for_baseline: By managing the stock and thinning them out we 're getting larger and better shellfish . '

source_bert_toks: by managing the stock and thin ##ning them out we ' re getting larger and better shell ##fish . '

reference_raw: By managing the stock and thinning it out, we're getting bigger and better molluscs.'

reference_spacy_tokens: By managing the stock and thinning it out , we 're getting bigger and better molluscs . '


fn_spacy_tokens_for_baseline: Sitting in front of the looking glass , making up her face , she hoped that Anthony was all right and not quite as heartbroken as Julia seemed to think he was .

source_bert_toks: sitting in front of the looking glass , making up her face , she hoped that anthony was all right and not quite as heart ##broken as julia seemed to think he was .

reference_raw: She sat in front of the mirror and made up her face, hoping that Anthony was okay and not as broken as Julie seemed to think.

reference_spacy_tokens: She sat in front of the mirror and made up her face , hoping that Anthony was okay and not as broken as Julie seemed to think .


fn_spacy_tokens_for_baseline: Zarathustra declared that at death God passes judgement on man and that this decides his fate when the world is finally transformed into the same state of perfection as when it left the hands of the Creator .

source_bert_toks: za ##rath ##ust ##ra declared that at death god passes judgement on man and that this decides his fate when the world is finally transformed into the same state of perfection as when it left the hands of the creator .

reference_raw: Zarathustra declared that at death God would give judgment to man and that it would decide his destiny when the world was finally transformed into the same state of perfection as when he left the hands of the Creator.

reference_spacy_tokens: Zarathustra declared that at death God would give judgment to man and that it would decide his destiny when the world was finally transformed into the same state of perfection as when he left the hands of the Creator .


fn_spacy_tokens_for_baseline: She drove an immaculately glossy green Beetle .

source_bert_toks: she drove an immaculate ##ly glossy green beetle .

reference_raw: She was driving a perfectly shiny green bug.

reference_spacy_tokens: She was driving a perfectly shiny green bug .


fn_spacy_tokens_for_baseline: Their interpretability , however , must not be exaggerated ; their meanings are not necessarily wholly predictable on first acquaintance .

source_bert_toks: their interpret ##ability , however , must not be exaggerated ; their meanings are not necessarily wholly predictable on first acquaintance .

reference_raw: However, their interpretation must not be exaggerated; their meaning is not necessarily entirely foreseeable at the time of first acquaintance.

reference_spacy_tokens: However , their interpretation must not be exaggerated ; their meaning is not necessarily entirely foreseeable at the time of first acquaintance .


fn_spacy_tokens_for_baseline: The significance of all this is that Utah had already convinced themselves that test - tube fusion could be an economic bonanza of unimaginable magnitude , and the first shots in the skirmish for priority had already been fired in February with the accusations that Jones had pirated the Utah chemists ' work .

source_bert_toks: the significance of all this is that utah had already convinced themselves that test - tube fusion could be an economic bon ##anza of un ##ima ##gina ##ble magnitude , and the first shots in the skirmish for priority had already been fired in february with the accusations that jones had pirate ##d the utah chemist ##s ' work .

reference_raw: The point is that Utah had already convinced himself that the fusion of the test tube might be an economic bonanza of unimaginable size, and the first shots in the fight of priority had already been fired in February on charges that Jones had pirated the work of the Utah chemists.

reference_spacy_tokens: The point is that Utah had already convinced himself that the fusion of the test tube might be an economic bonanza of unimaginable size , and the first shots in the fight of priority had already been fired in February on charges that Jones had pirated the work of the Utah chemists .


fn_spacy_tokens_for_baseline: The Association had become well known during the occupations , and its membership increased dramatically .

source_bert_toks: the association had become well known during the occupations , and its membership increased dramatically .

reference_raw: The Association has become well-known in the course of its occupations and has grown dramatically.

reference_spacy_tokens: The Association has become well - known in the course of its occupations and has grown dramatically .


fn_spacy_tokens_for_baseline: Hall 's landmark visit to Iraq started on Sunday evening when he arrived here by land from Amman , Jordan .

source_bert_toks: hall ' s landmark visit to iraq started on sunday evening when he arrived here by land from amman , jordan .

reference_raw: Hall's visit to Iraq began on Sunday night when he arrived on land from Amman, Jordan.

reference_spacy_tokens: Hall 's visit to Iraq began on Sunday night when he arrived on land from Amman , Jordan .


fn_spacy_tokens_for_baseline: Athelstan , however , informed them his companion was no less a personage than Sir John Cranston , who was now ill , so they stepped aside , doing their best to hide their smirks .

source_bert_toks: at ##hel ##stan , however , informed them his companion was no less a persona ##ge than sir john cr ##ans ##ton , who was now ill , so they stepped aside , doing their best to hide their smirk ##s .

reference_raw: Athelstan informed them, however, that his companion was no less a person than Sir John Cranston, who was now sick, so they stepped back and did their best to hide their smiles.

reference_spacy_tokens: Athelstan informed them , however , that his companion was no less a person than Sir John Cranston , who was now sick , so they stepped back and did their best to hide their smiles .


fn_spacy_tokens_for_baseline: Also in 1989 , the State Trading Corporation of India admitted that it had sold Iran over 60 tons of thionyl chloride ( a nerve agent precursor ) and that its supplier was planning to ship an additional 257 tons of the chemical to Iran .

source_bert_toks: also in 1989 , the state trading corporation of india admitted that it had sold iran over 60 tons of th ##ion ##yl chloride ( a nerve agent precursor ) and that its supplier was planning to ship an additional 257 tons of the chemical to iran .

reference_raw: In 1989, the Indian State Trading Corporation also acknowledged that it had sold more than 60 tonnes of thionyl chloride (a precursor of nerve agents) to Iran and that its supplier intended to send another 257 tonnes of that chemical to Iran.

reference_spacy_tokens: In 1989 , the Indian State Trading Corporation also acknowledged that it had sold more than 60 tonnes of thionyl chloride ( a precursor of nerve agents ) to Iran and that its supplier intended to send another 257 tonnes of that chemical to Iran .


fn_spacy_tokens_for_baseline: According to Edward Altman , a finance professor at New York University , there are also indirect costs : some customers choose not to deal with vulnerable firms , and suppliers demand better terms .

source_bert_toks: according to edward alt ##man , a finance professor at new york university , there are also indirect costs : some customers choose not to deal with vulnerable firms , and suppliers demand better terms .

reference_raw: According to Edward Altman, a financial teacher at the University of New York, there are indirect costs: some customers choose not to deal with vulnerable companies, and suppliers demand better conditions.

reference_spacy_tokens: According to Edward Altman , a financial teacher at the University of New York , there are indirect costs : some customers choose not to deal with vulnerable companies , and suppliers demand better conditions .


fn_spacy_tokens_for_baseline: Finally , the fact that the GAG and receptor binding regions are on opposite regions of the PEDF molecule , suggest the possibility of the existence of PEDF mutants or engineered variants that , having lost or decreased GAG - binding capabilities , still show high affinity for the PEDF receptor in a way not dependent upon GAGs content or composition .

source_bert_toks: finally , the fact that the gag and receptor binding regions are on opposite regions of the pe ##df molecule , suggest the possibility of the existence of pe ##df mutants or engineered variants that , having lost or decreased gag - binding capabilities , still show high affinity for the pe ##df receptor in a way not dependent upon gag ##s content or composition .

reference_raw: Finally, the fact that the GAG and receptor binding regions are located in opposite regions of the PEDF molecule suggests that there may be PEDF mutants or engineered variants which, after loss or reduction of the GAG binding capacity, still have a high affinity to the PEDF receptor in a way that does not depend on the content or composition of the GAGs.

reference_spacy_tokens: Finally , the fact that the GAG and receptor binding regions are located in opposite regions of the PEDF molecule suggests that there may be PEDF mutants or engineered variants which , after loss or reduction of the GAG binding capacity , still have a high affinity to the PEDF receptor in a way that does not depend on the content or composition of the GAGs .


fn_spacy_tokens_for_baseline: Philip , as overlord of Brittany , at once claimed custody of Geoffrey 's two daughters , and threatened to invade Normandy if Henry II did not hand them over .

source_bert_toks: philip , as over ##lord of brittany , at once claimed custody of geoffrey ' s two daughters , and threatened to invade normandy if henry ii did not hand them over .

reference_raw: Philip, Lord of Brittany, immediately claimed custody of Geoffrey's two daughters and threatened to attack Normandy unless Henry II gave them to him.

reference_spacy_tokens: Philip , Lord of Brittany , immediately claimed custody of Geoffrey 's two daughters and threatened to attack Normandy unless Henry II gave them to him .


fn_spacy_tokens_for_baseline: But Iris sprang up , hit him and shouted : ' Get lost , you thieving swine . '

source_bert_toks: but iris sprang up , hit him and shouted : ' get lost , you th ##ieving sw ##ine . '

reference_raw: But Iris jumped to his feet, struck him, and shouted, "Get lost, you thieving pig."

reference_spacy_tokens: But Iris jumped to his feet , struck him , and shouted , " Get lost , you thieving pig . "


fn_spacy_tokens_for_baseline: In 1855 , Mormon leader Brigham Young responded to promising reports of Las Vegas by sending 30 missionary settlers to the valley ; they eventually built a fort not far from today 's Downtown .

source_bert_toks: in 1855 , mormon leader brigham young responded to promising reports of las vegas by sending 30 missionary settlers to the valley ; they eventually built a fort not far from today ' s downtown .

reference_raw: In 1855, the Mormon leader, Brigham Young, responded to the promising news of Las Vegas by sending 30 missionary settlers into the valley, eventually building a fortress not far from the Downtown.

reference_spacy_tokens: In 1855 , the Mormon leader , Brigham Young , responded to the promising news of Las Vegas by sending 30 missionary settlers into the valley , eventually building a fortress not far from the Downtown .


fn_spacy_tokens_for_baseline: Traffic chaos as lorry spills grain in crash

source_bert_toks: traffic chaos as lo ##rry spill ##s grain in crash

reference_raw: Traffic confusion when a truck spills grain in a crash


fn_spacy_tokens_for_baseline: With unemployment at its lowest point in years in central Indiana , many people still have a hard time finding meaningful employment .

source_bert_toks: with unemployment at its lowest point in years in central indiana , many people still have a hard time finding meaningful employment .

reference_raw: With unemployment at the lowest point in the years in Central Indiana, many people still find it difficult to find meaningful jobs.

reference_spacy_tokens: With unemployment at the lowest point in the years in Central Indiana , many people still find it difficult to find meaningful jobs .


fn_spacy_tokens_for_baseline: It must be very rewarding to be in a position to watch your granddaughter grow up , especially in such a gorgeous spot . '

source_bert_toks: it must be very reward ##ing to be in a position to watch your granddaughter grow up , especially in such a gorgeous spot . '

reference_raw: It must be very rewarding to be able to watch your grandchild grow up, especially in such a beautiful place.'

reference_spacy_tokens: It must be very rewarding to be able to watch your grandchild grow up , especially in such a beautiful place . '


fn_spacy_tokens_for_baseline: There is a bewildering array of environmental and wildlife holidays to choose from .

source_bert_toks: there is a be ##wil ##der ##ing array of environmental and wildlife holidays to choose from .

reference_raw: There's a confusing array of environmental and wildlife holidays to choose from.

reference_spacy_tokens: There 's a confusing array of environmental and wildlife holidays to choose from .


fn_spacy_tokens_for_baseline: I paint my face , cover myself in beads .

source_bert_toks: i paint my face , cover myself in beads .

reference_raw: I'm painting my cheeks and covering myself with beads.

reference_spacy_tokens: I 'm painting my cheeks and covering myself with beads .


fn_spacy_tokens_for_baseline: A popular optional excursion is an hour 's detour to Guangzhou Zoo , founded in 1958 .

source_bert_toks: a popular optional excursion is an hour ' s det ##our to guangzhou zoo , founded in 1958 .

reference_raw: The popular optional trip is an hour's detour to the Guangzhou Zoo, founded in 1958.

reference_spacy_tokens: The popular optional trip is an hour 's detour to the Guangzhou Zoo , founded in 1958 .


fn_spacy_tokens_for_baseline: ' You do n't have to be haunted by the past -- or by that abominable old sod upstairs .

source_bert_toks: ' you don ' t have to be haunted by the past - - or by that ab ##omi ##nable old so ##d upstairs .

reference_raw: 'You don't need to be haunted by the past - or by that disgusting old sod up there.

reference_spacy_tokens: ' You do n't need to be haunted by the past - or by that disgusting old sod up there .


fn_spacy_tokens_for_baseline: The opportunities for commercial exploitation of the hiking boom were grasped readily by the railway companies who offered cheap day returns to hikers .

source_bert_toks: the opportunities for commercial exploitation of the hiking boom were grasped readily by the railway companies who offered cheap day returns to hike ##rs .

reference_raw: Rail companies offering low-cost daily returns to tourists took advantage of the opportunities for commercial exploitation of the tourist boom.

reference_spacy_tokens: Rail companies offering low - cost daily returns to tourists took advantage of the opportunities for commercial exploitation of the tourist boom .


fn_spacy_tokens_for_baseline: Ronni had rarely tasted anything quite so delicious .

source_bert_toks: ron ##ni had rarely tasted anything quite so delicious .

reference_raw: Ronni seldom felt anything so tasty.

reference_spacy_tokens: Ronni seldom felt anything so tasty .


fn_spacy_tokens_for_baseline: These had been her favourites when she was little but had not of ten been allowed because the sauce oozed out the sides and made a mess of your clothes .

source_bert_toks: these had been her favourites when she was little but had not of ten been allowed because the sauce o ##oz ##ed out the sides and made a mess of your clothes .

reference_raw: These were her favorites when she was a little girl, but she wasn't allowed to be ten, because the sauce was dripping off her hips and messing up your clothes.

reference_spacy_tokens: These were her favorites when she was a little girl , but she was n't allowed to be ten , because the sauce was dripping off her hips and messing up your clothes .


fn_spacy_tokens_for_baseline: Each and every June , the world 's top amateur cyclists get on their bikes to pedal the roads , lanes , hills and mountains of Britain .

source_bert_toks: each and every june , the world ' s top amateur cyclists get on their bikes to pedal the roads , lanes , hills and mountains of britain .

reference_raw: Every June, the best amateur cyclists in the world get on their bikes and walk the streets, lanes, hills, and mountains of Britain.

reference_spacy_tokens: Every June , the best amateur cyclists in the world get on their bikes and walk the streets , lanes , hills , and mountains of Britain .


fn_spacy_tokens_for_baseline: It seems that when they were made to lie down in the back of the truck and their hands were tied behind them , Katrine remembers there being rags underneath her .

source_bert_toks: it seems that when they were made to lie down in the back of the truck and their hands were tied behind them , kat ##rine remembers there being rags underneath her .

reference_raw: Apparently, when they were forced to lie in the back of the car with their hands tied behind them, Katrine remembers that there are rags under it.

reference_spacy_tokens: Apparently , when they were forced to lie in the back of the car with their hands tied behind them , Katrine remembers that there are rags under it .


fn_spacy_tokens_for_baseline: He is either a foreigner or has lived long in the tropics , for he is yellow and sapless , but tough as whipcord .

source_bert_toks: he is either a foreigner or has lived long in the tr ##op ##ics , for he is yellow and sap ##less , but tough as whip ##cor ##d .

reference_raw: Either he's a stranger, or he's lived long in the tropics, because he's yellow and sapless, but he's hard as a whipcord.

reference_spacy_tokens: Either he 's a stranger , or he 's lived long in the tropics , because he 's yellow and sapless , but he 's hard as a whipcord .


fn_spacy_tokens_for_baseline: The other mansions belonged to prosaic and respectable people who live far aloof from romance .

source_bert_toks: the other mansions belonged to pro ##sai ##c and respectable people who live far al ##oof from romance .

reference_raw: The rest of the estate was owned by prosaic and respectable people who lived far away from romance.

reference_spacy_tokens: The rest of the estate was owned by prosaic and respectable people who lived far away from romance .


fn_spacy_tokens_for_baseline: Table bookings and individual tickets , which include a champagne reception sponsored by Penguin and a four - course dinner , cost £ 35 including VAT .

source_bert_toks: table booking ##s and individual tickets , which include a champagne reception sponsored by penguin and a four - course dinner , cost £3 ##5 including va ##t .

reference_raw: Table reservations and individual tickets, including a Penguin-sponsored champagne banquet and a four-course dinner, cost £35, including VAT.

reference_spacy_tokens: Table reservations and individual tickets , including a Penguin - sponsored champagne banquet and a four - course dinner , cost £ 35 , including VAT .


fn_spacy_tokens_for_baseline: The absence of any clear cut environmental influences on the variable hepatic response to alcohol has recently diverted attention towards a search for genetic factors that might offer an explanation .

source_bert_toks: the absence of any clear cut environmental influences on the variable he ##pati ##c response to alcohol has recently diverted attention towards a search for genetic factors that might offer an explanation .

reference_raw: The lack of a clear reduction in the environmental impact on the variable liver response to alcohol has recently diverted attention to finding genetic factors that could provide explanations.

reference_spacy_tokens: The lack of a clear reduction in the environmental impact on the variable liver response to alcohol has recently diverted attention to finding genetic factors that could provide explanations .


fn_spacy_tokens_for_baseline: But gratitude was all she felt , and even that was overshadowed by anger at the way he had blackmailed her into sailing with him .

source_bert_toks: but gratitude was all she felt , and even that was overshadowed by anger at the way he had blackmail ##ed her into sailing with him .

reference_raw: But gratitude was all she felt, and even that was overshadowed by anger at how he had extorted her into sailing with him.

reference_spacy_tokens: But gratitude was all she felt , and even that was overshadowed by anger at how he had extorted her into sailing with him .


fn_spacy_tokens_for_baseline: The captain gave him a bowl of cold stew which he could only swallow by gulping it down with wine .

source_bert_toks: the captain gave him a bowl of cold stew which he could only swallow by gulp ##ing it down with wine .

reference_raw: The captain handed him a plate of cold stew that he could swallow only by swallowing it with wine.

reference_spacy_tokens: The captain handed him a plate of cold stew that he could swallow only by swallowing it with wine .


fn_spacy_tokens_for_baseline: Contravention of ss.1 , 2 , 4 , or 5 of the CSA 1985 makes an individual liable to criminal penalty .

source_bert_toks: contra ##vent ##ion of ss . 1 , 2 , 4 , or 5 of the csa 1985 makes an individual liable to criminal penalty .

reference_raw: The violation of SS.1, 2, 4 or 5 of the CSA 1985 makes a person liable for a criminal sanction.

reference_spacy_tokens: The violation of SS.1 , 2 , 4 or 5 of the CSA 1985 makes a person liable for a criminal sanction .


fn_spacy_tokens_for_baseline: We had no boots or gloves , and my hands and feet ached badly .

source_bert_toks: we had no boots or gloves , and my hands and feet ached badly .

reference_raw: We didn't have shoes or gloves, and my hands and feet hurt.

reference_spacy_tokens: We did n't have shoes or gloves , and my hands and feet hurt .


fn_spacy_tokens_for_baseline: He tripped backwards up the steps .

source_bert_toks: he tripped backwards up the steps .

reference_raw: He stumbled back up the stairs.

reference_spacy_tokens: He stumbled back up the stairs .


fn_spacy_tokens_for_baseline: She stood a moment in the sun , pulling her veils around her in one swift movement .

source_bert_toks: she stood a moment in the sun , pulling her veil ##s around her in one swift movement .

reference_raw: She stood in the sun for a moment, pulling the curtains around her in one quick motion.

reference_spacy_tokens: She stood in the sun for a moment , pulling the curtains around her in one quick motion .


fn_spacy_tokens_for_baseline: The renewed investigation of Morsi and members of the Muslim Brotherhood dates to the uprising that led to the ouster of Mubarak in 2011 .

source_bert_toks: the renewed investigation of mor ##si and members of the muslim brotherhood dates to the uprising that led to the ou ##ster of mu ##bara ##k in 2011 .

reference_raw: The re-investigation of Morsi and members of the Muslim Brotherhood dates back to the uprising that led to Mubarak's outburst in 2011.

reference_spacy_tokens: The re - investigation of Morsi and members of the Muslim Brotherhood dates back to the uprising that led to Mubarak 's outburst in 2011 .


fn_spacy_tokens_for_baseline: The left also is critical of the style of the Socialist government -- a remarkable parallel to the situation in Britain .

source_bert_toks: the left also is critical of the style of the socialist government - - a remarkable parallel to the situation in britain .

reference_raw: The right also criticizes the style of the socialist government - a remarkable parallel to the situation in Britain.

reference_spacy_tokens: The right also criticizes the style of the socialist government - a remarkable parallel to the situation in Britain .


fn_spacy_tokens_for_baseline: THE PUBLIC AS as well as the practising artist , whether amateur or professional , seems always intrigued by the ways that another artist has arrived at being professionally and financially independent .

source_bert_toks: the public as as well as the pr ##act ##ising artist , whether amateur or professional , seems always intrigued by the ways that another artist has arrived at being professionally and financially independent .

reference_raw: PUBLIC AS and practising artists, whether amateur or professional, always seem to be interested in the way another artist has come to be professionally and financially independent.

reference_spacy_tokens: PUBLIC AS and practising artists , whether amateur or professional , always seem to be interested in the way another artist has come to be professionally and financially independent .


fn_spacy_tokens_for_baseline: Ashamed of having frightened me , he looked at me sweetly and began to sing Italian songs to make me forget the incident ' .

source_bert_toks: ashamed of having frightened me , he looked at me sweetly and began to sing italian songs to make me forget the incident ' .

reference_raw: Shamed that he had frightened me, he gave me a sweet look and began to sing Italian songs to make me forget about the incident.

reference_spacy_tokens: Shamed that he had frightened me , he gave me a sweet look and began to sing Italian songs to make me forget about the incident .


fn_spacy_tokens_for_baseline: As for me , I 'll stick with my philosophy of reporting and , I guess , life -- nobody knows .

source_bert_toks: as for me , i ' ll stick with my philosophy of reporting and , i guess , life - - nobody knows .

reference_raw: As far as I'm concerned, I'm gonna stick to my philosophy of reporting and, I think, life -- no one knows.

reference_spacy_tokens: As far as I 'm concerned , I 'm gon na stick to my philosophy of reporting and , I think , life -- no one knows .


fn_spacy_tokens_for_baseline: ' I wish you 'd shut up about Chris , ' said Camille .

source_bert_toks: ' i wish you ' d shut up about chris , ' said camille .

reference_raw: 'I'd like you to keep quiet about Chris,' Camille said.

reference_spacy_tokens: ' I 'd like you to keep quiet about Chris , ' Camille said .


fn_spacy_tokens_for_baseline: Emily Davidson died under the hooves of the King 's horse .

source_bert_toks: emily davidson died under the hooves of the king ' s horse .

reference_raw: Emily Davidson died under the King's horse's hoof.

reference_spacy_tokens: Emily Davidson died under the King 's horse 's hoof .


fn_spacy_tokens_for_baseline: The Minoans did not invent the technique of carving shell : it was used earlier , in the Cyclades , for making figurines and bracelets .

source_bert_toks: the min ##oa ##ns did not in ##vent the technique of carving shell : it was used earlier , in the cy ##cl ##ades , for making fig ##uri ##nes and bracelet ##s .

reference_raw: The Minoans had not devised the technique of carving shells: they had been used before in the Cyclades to make statues and bracelets.

reference_spacy_tokens: The Minoans had not devised the technique of carving shells : they had been used before in the Cyclades to make statues and bracelets .


fn_spacy_tokens_for_baseline: Water , foam and clothes gushed out of the machine on to the floor .

source_bert_toks: water , foam and clothes gus ##hed out of the machine on to the floor .

reference_raw: Water, foam, and clothing poured from the machine to the floor.

reference_spacy_tokens: Water , foam , and clothing poured from the machine to the floor .


fn_spacy_tokens_for_baseline: Finn was an inexperienced and inexpert puppeteer .

source_bert_toks: finn was an inexperienced and in ##ex ##per ##t puppet ##eer .

reference_raw: Finn was an inexperienced and inexperienced puppeteer.

reference_spacy_tokens: Finn was an inexperienced and inexperienced puppeteer .


fn_spacy_tokens_for_baseline: When Charles appeared for dinner , Diana noticed that he was sporting a new pair of cufflinks in the shape of two ' C 's intertwined .

source_bert_toks: when charles appeared for dinner , diana noticed that he was sporting a new pair of cuff ##link ##s in the shape of two ' c ' s inter ##twined .

reference_raw: When Charles showed up for dinner, Diana noticed that he was sporting a new pair of handcuffs in the shape of two 'C's.

reference_spacy_tokens: When Charles showed up for dinner , Diana noticed that he was sporting a new pair of handcuffs in the shape of two ' C 's .


fn_spacy_tokens_for_baseline: I had been seeing Jessica for twelve months before she admitted to herself that her lover would never come back .

source_bert_toks: i had been seeing jessica for twelve months before she admitted to herself that her lover would never come back .

reference_raw: I spent twelve months with Jessica before she admitted that her lover would never return.

reference_spacy_tokens: I spent twelve months with Jessica before she admitted that her lover would never return .


fn_spacy_tokens_for_baseline: Over the past 10 years AI has documented more than 4000 cases of ' disappeared ' people in Peru , scores of summary executions and at least 15 massacres by the security forces .

source_bert_toks: over the past 10 years ai has documented more than 4000 cases of ' disappeared ' people in peru , scores of summary executions and at least 15 massacre ##s by the security forces .

reference_raw: Over the last 10 years, AI has documented over 4,000 cases of "disappeared" people in Peru, dozens of summary executions, and at least 15 security carnages.

reference_spacy_tokens: Over the last 10 years , AI has documented over 4,000 cases of " disappeared " people in Peru , dozens of summary executions , and at least 15 security carnages .


fn_spacy_tokens_for_baseline: The Brigadier set down a fat puppy that he had been holding and squelched towards the yard , driving a dozen pullets before him .

source_bert_toks: the brigadier set down a fat puppy that he had been holding and sq ##uel ##ched towards the yard , driving a dozen pull ##ets before him .

reference_raw: The brigadier put down the fat puppy he had been holding and squealed toward the courtyard, driving a dozen pullets in front of him.

reference_spacy_tokens: The brigadier put down the fat puppy he had been holding and squealed toward the courtyard , driving a dozen pullets in front of him .


fn_spacy_tokens_for_baseline: The first Barrett family member came to Jamaica with the invading English forces .

source_bert_toks: the first barrett family member came to jamaica with the invading english forces .

reference_raw: The first Barrett family member came to Jamaica with an invasion of English troops.

reference_spacy_tokens: The first Barrett family member came to Jamaica with an invasion of English troops .


fn_spacy_tokens_for_baseline: All the talk of weddings made Anne feel downhearted and impatient with the secrecy that John insisted on .

source_bert_toks: all the talk of weddings made anne feel down ##hearted and impatient with the secrecy that john insisted on .

reference_raw: All the talk of weddings made Anne feel heartfelt and impatient with the secret John had insisted on.

reference_spacy_tokens: All the talk of weddings made Anne feel heartfelt and impatient with the secret John had insisted on .


fn_spacy_tokens_for_baseline: This effort to unite what can not be united lies at the heart of the haiku and accounts for some of its tense , sad loveliness .

source_bert_toks: this effort to unite what can not be united lies at the heart of the hai ##ku and accounts for some of its tense , sad love ##liness .

reference_raw: This effort to unify what cannot be unified lies at the heart of the haik and explains some of its strained, sad beauty.

reference_spacy_tokens: This effort to unify what can not be unified lies at the heart of the haik and explains some of its strained , sad beauty .


fn_spacy_tokens_for_baseline: American education 's future will be determined by the degree to which we -- all of us -- allow this common sense to prevail over the illusions that we also share .

source_bert_toks: american education ' s future will be determined by the degree to which we - - all of us - - allow this common sense to pre ##va ##il over the illusions that we also share .

reference_raw: The future of American education will depend on the extent to which we, all of us, let this common sense prevail over the illusions we share.

reference_spacy_tokens: The future of American education will depend on the extent to which we , all of us , let this common sense prevail over the illusions we share .


fn_spacy_tokens_for_baseline: Two pellets hit a seven - year - old girl playing outside .

source_bert_toks: two pe ##llet ##s hit a seven - year - old girl playing outside .

reference_raw: Two bullets struck a seven-year-old girl playing outside.

reference_spacy_tokens: Two bullets struck a seven - year - old girl playing outside .


fn_spacy_tokens_for_baseline: Jehana had thrown off her cloak , and now she took her hiranu and cut a wide swathe from her gown .

source_bert_toks: je ##hana had thrown off her cloak , and now she took her hi ##ran ##u and cut a wide swat ##he from her gown .

reference_raw: Jehana dropped her cloak and took the hiranu and cut a wide swathe from her robe.

reference_spacy_tokens: Jehana dropped her cloak and took the hiranu and cut a wide swathe from her robe .


fn_spacy_tokens_for_baseline: Gardens , particularly flower gardens , serve no real purpose .

source_bert_toks: gardens , particularly flower gardens , serve no real purpose .

reference_raw: The gardens, especially the flower gardens, have no real purpose.

reference_spacy_tokens: The gardens , especially the flower gardens , have no real purpose .


fn_spacy_tokens_for_baseline: She looked down and shuffled her feet inanely .

source_bert_toks: she looked down and shuffled her feet ina ##nel ##y .

reference_raw: She looked down and shook her legs uneasily.

reference_spacy_tokens: She looked down and shook her legs uneasily .


fn_spacy_tokens_for_baseline: He was n't going to put up with this complete lack of concern a moment longer .

source_bert_toks: he wasn ' t going to put up with this complete lack of concern a moment longer .

reference_raw: He couldn't bear the total lack of concern any longer.

reference_spacy_tokens: He could n't bear the total lack of concern any longer .


fn_spacy_tokens_for_baseline: An early interpretation saw them as barracks for troops used by Swegen Forkbeard to conquer England , with each house containing a ship 's crew .

source_bert_toks: an early interpretation saw them as barracks for troops used by sw ##ege ##n fork ##be ##ard to conquer england , with each house containing a ship ' s crew .

reference_raw: The first interpretation saw them as barracks for the soldiers used by the Swegen Forkbeard to take England, each house containing the crew of the ship.

reference_spacy_tokens: The first interpretation saw them as barracks for the soldiers used by the Swegen Forkbeard to take England , each house containing the crew of the ship .


fn_spacy_tokens_for_baseline: He had heard shouts like these then , and the bark of dogs , and the cries and screams as Jewish friends near by were found and summarily shot .

source_bert_toks: he had heard shouts like these then , and the bark of dogs , and the cries and screams as jewish friends near by were found and sum ##mar ##ily shot .

reference_raw: He had heard such screams at the time, and heard the screams of dogs, and the screams and screams of Jewish friends nearby, found and summarily shot.

reference_spacy_tokens: He had heard such screams at the time , and heard the screams of dogs , and the screams and screams of Jewish friends nearby , found and summarily shot .


fn_spacy_tokens_for_baseline: Also worth investigating are contemporary handicrafts , both Portuguese and Chinese , from across the border .

source_bert_toks: also worth investigating are contemporary hand ##ic ##raf ##ts , both portuguese and chinese , from across the border .

reference_raw: Contemporary handicrafts, both Portuguese and Chinese, from abroad are also worth investigating.

reference_spacy_tokens: Contemporary handicrafts , both Portuguese and Chinese , from abroad are also worth investigating .


fn_spacy_tokens_for_baseline: In short , inveighing against hierarchy is rhetoric : both the logic of efficiency and the historical evidence disclose that non - hierarchical modes are mainly of ephemeral duration . ( 1980 : 35 ) .

source_bert_toks: in short , in ##ve ##igh ##ing against hierarchy is rhetoric : both the logic of efficiency and the historical evidence disclose that non - hierarchical modes are mainly of ep ##hem ##eral duration . ( 1980 : 35 ) .

reference_raw: In short, investigations against hierarchy are rhetorical: both efficiency logic and historical evidence show that non-hierarchical modes have a predominantly fleeting duration (1980: 35).

reference_spacy_tokens: In short , investigations against hierarchy are rhetorical : both efficiency logic and historical evidence show that non - hierarchical modes have a predominantly fleeting duration ( 1980 : 35 ) .


fn_spacy_tokens_for_baseline: The inevitable slowing down in drilling which will stem from the withdrawal of the concession is bound to hit jobs at Scottish supply bases sooner rather than later .

source_bert_toks: the inevitable slowing down in drilling which will stem from the withdrawal of the concession is bound to hit jobs at scottish supply bases sooner rather than later .

reference_raw: The inevitable slowdown in drilling resulting from the cancellation of the concession will inevitably hit jobs at Scottish supply bases sooner rather than later.

reference_spacy_tokens: The inevitable slowdown in drilling resulting from the cancellation of the concession will inevitably hit jobs at Scottish supply bases sooner rather than later .


fn_spacy_tokens_for_baseline: We were two and a half hours over guidebook time : so much for the armchair boasts of perfect stamina !

source_bert_toks: we were two and a half hours over guide ##book time : so much for the armchair boasts of perfect st ##amina !

reference_raw: We were two-and-a-half hours away from guide time: so much for the chairs to brag about perfect stamina!

reference_spacy_tokens: We were two - and - a - half hours away from guide time : so much for the chairs to brag about perfect stamina !


fn_spacy_tokens_for_baseline: As the pope strapped the sword round the emperor 's waist , he reminded the emperor that the saints had conquered kingdoms not by the sword but by faith .

source_bert_toks: as the pope strapped the sword round the emperor ' s waist , he reminded the emperor that the saints had conquered kingdoms not by the sword but by faith .

reference_raw: When the Pope put his sword around the Emperor's waist, he reminded the Emperor that the saints had conquered kingdoms not by sword but by faith.

reference_spacy_tokens: When the Pope put his sword around the Emperor 's waist , he reminded the Emperor that the saints had conquered kingdoms not by sword but by faith .


fn_spacy_tokens_for_baseline: 2.30AM Outside a gutted supermarket , I stop to talk to a group of young men pelting two carloads of police with pilfered eggs .

source_bert_toks: 2 . 30 ##am outside a gut ##ted supermarket , i stop to talk to a group of young men pe ##lt ##ing two carlo ##ads of police with pi ##lf ##ered eggs .

reference_raw: 2.30 A.M. In front of the gutted supermarket, I stopped to talk to a group of young men who were pelting two loads of police with pilfered eggs.

reference_spacy_tokens: 2.30 A.M. In front of the gutted supermarket , I stopped to talk to a group of young men who were pelting two loads of police with pilfered eggs .


fn_spacy_tokens_for_baseline: Roast the meat 15 - 20 minutes , when it will still be pink .

source_bert_toks: roast the meat 15 - 20 minutes , when it will still be pink .

reference_raw: Reast the meat in 15 to 20 minutes while it's still pink.

reference_spacy_tokens: Reast the meat in 15 to 20 minutes while it 's still pink .


fn_spacy_tokens_for_baseline: All were charged under the National Security Law which provided for a maximum penalty of death by hanging .

source_bert_toks: all were charged under the national security law which provided for a maximum penalty of death by hanging .

reference_raw: They were all charged under the National Security Act, which laid down the maximum sentence of death by hanging.

reference_spacy_tokens: They were all charged under the National Security Act , which laid down the maximum sentence of death by hanging .


fn_spacy_tokens_for_baseline: In most cases , these symptoms persist for a couple of days .

source_bert_toks: in most cases , these symptoms persist for a couple of days .

reference_raw: In most cases, these symptoms last for several days.

reference_spacy_tokens: In most cases , these symptoms last for several days .


fn_spacy_tokens_for_baseline: He should consider his victory a mandate to continue his growth - oriented economic reforms and not a demand that he move further left .

source_bert_toks: he should consider his victory a mandate to continue his growth - oriented economic reforms and not a demand that he move further left .

reference_raw: He should consider his victory a task to continue his growth-oriented economic reforms, not a demand to go further to the left.

reference_spacy_tokens: He should consider his victory a task to continue his growth - oriented economic reforms , not a demand to go further to the left .


fn_spacy_tokens_for_baseline: On Jan. 29 two Iraqi battalion - sized armoured forces invaded Saudi Arabia .

source_bert_toks: on jan . 29 two iraqi battalion - sized armoured forces invaded saudi arabia .

reference_raw: On January 29, two Iraqi battalion-sized armed forces attacked Saudi Arabia.

reference_spacy_tokens: On January 29 , two Iraqi battalion - sized armed forces attacked Saudi Arabia .


fn_spacy_tokens_for_baseline: He tramped the moors every day .

source_bert_toks: he tram ##ped the moors every day .

reference_raw: Every day he trampled the docks.

reference_spacy_tokens: Every day he trampled the docks .


fn_spacy_tokens_for_baseline: A licence will normally start on the date on which a valid application is received or after the expiry of the previous licence if later .

source_bert_toks: a licence will normally start on the date on which a valid application is received or after the ex ##pi ##ry of the previous licence if later .

reference_raw: The licence shall normally begin on the date of receipt of the current application or, if later, on the expiry of the previous licence.

reference_spacy_tokens: The licence shall normally begin on the date of receipt of the current application or , if later , on the expiry of the previous licence .


fn_spacy_tokens_for_baseline: And in one poll 29 % of blacks were willing to countenance the idea that AIDS was created by whites to eradicate blacks .

source_bert_toks: and in one poll 29 % of blacks were willing to count ##ena ##nce the idea that aids was created by whites to era ##dicate blacks .

reference_raw: And in one poll, 29% of black people were prepared to face the idea that white people created AIDS to eradicate black people.

reference_spacy_tokens: And in one poll , 29 % of black people were prepared to face the idea that white people created AIDS to eradicate black people .


fn_spacy_tokens_for_baseline: " Potential foreign investors are thirsty for news about Africa and for signs of openess by governments , " the letter said .

source_bert_toks: " potential foreign investors are thirsty for news about africa and for signs of open ##ess by governments , " the letter said .

reference_raw: "Potential foreign investors thirst for news of Africa and signs of government openness," the letter said.

reference_spacy_tokens: " Potential foreign investors thirst for news of Africa and signs of government openness , " the letter said .


fn_spacy_tokens_for_baseline: It was Blackberry who bullied the stupefied Pipkin to his feet and forced him to limp the few yards to the gravel spit .

source_bert_toks: it was blackberry who bull ##ied the stu ##pe ##fied pip ##kin to his feet and forced him to limp the few yards to the gravel spit .

reference_raw: It was Blackberry who pushed the stupefied Pipkin to his feet and made him limp a few yards to the gravel spit.

reference_spacy_tokens: It was Blackberry who pushed the stupefied Pipkin to his feet and made him limp a few yards to the gravel spit .


fn_spacy_tokens_for_baseline: Not only would this permit the benefits of specialisation and economies of scale ( ie lower average cost ) to be felt , but would also avoid expensive duplication .

source_bert_toks: not only would this permit the benefits of special ##isation and economies of scale ( ie lower average cost ) to be felt , but would also avoid expensive du ##plication .

reference_raw: This would not only allow the benefits of specialisation and economies of scale (i.e. lower average costs) to be felt, but would also avoid costly overlaps.

reference_spacy_tokens: This would not only allow the benefits of specialisation and economies of scale ( i.e. lower average costs ) to be felt , but would also avoid costly overlaps .