fn_spacy_tokens_for_baseline: This means that the Directory must be able to handle national characters not found in the US - ASCII repetoire .

source_bert_toks: this means that the directory must be able to handle national characters not found in the us - as ##ci ##i rep ##eto ##ire .

reference_raw: This means that the directory must be capable of handling national characters that are not present in the US-ASCII repetoire.

reference_spacy_tokens: This means that the directory must be capable of handling national characters that are not present in the US - ASCII repetoire .


fn_spacy_tokens_for_baseline: Unfortunately , those same charitable organizations , those same charity committees that funnel the funds to needier Palestinians , also serve to support terrorist attacks .

source_bert_toks: unfortunately , those same charitable organizations , those same charity committees that funnel the funds to need ##ier palestinians , also serve to support terrorist attacks .

reference_raw: Unfortunately, the same charitable organizations, the same charitable committees that finance the needy Palestinians, are also used to support terrorist attacks.

reference_spacy_tokens: Unfortunately , the same charitable organizations , the same charitable committees that finance the needy Palestinians , are also used to support terrorist attacks .


fn_spacy_tokens_for_baseline: Iran was seeking fuel - fabrication technology due to previous supply problems with the safeguarded low enriched uranium fuel shipped from Argentina ( under a previous contract ) for a research reactor supplied by the US in 1967 .

source_bert_toks: iran was seeking fuel - fabrication technology due to previous supply problems with the safeguard ##ed low enriched uranium fuel shipped from argentina ( under a previous contract ) for a research reactor supplied by the us in 1967 .

reference_raw: Iran sought fuel - production technology because of previous supply problems with secure, low-enriched uranium fuel shipped from Argentina (under a previous contract) to a research reactor supplied by the United States in 1967.

reference_spacy_tokens: Iran sought fuel - production technology because of previous supply problems with secure , low - enriched uranium fuel shipped from Argentina ( under a previous contract ) to a research reactor supplied by the United States in 1967 .


fn_spacy_tokens_for_baseline: ' It will not be possible for you to present in that compact form which is dear to your heart .

source_bert_toks: ' it will not be possible for you to present in that compact form which is dear to your heart .

reference_raw: 'You won't be able to show up in the compact form that's dear to your heart.

reference_spacy_tokens: ' You wo n't be able to show up in the compact form that 's dear to your heart .


fn_spacy_tokens_for_baseline: ' We do n't go to Lightwater Valley , ' said an equally affronted bus driver .

source_bert_toks: ' we don ' t go to light ##water valley , ' said an equally af ##front ##ed bus driver .

reference_raw: 'We're not going to the Valley of Lightwater,' said the equally offended bus driver.

reference_spacy_tokens: ' We 're not going to the Valley of Lightwater , ' said the equally offended bus driver .


fn_spacy_tokens_for_baseline: We can warehouse your merchandise , repack and consolidate it , deliver it to retailers .

source_bert_toks: we can warehouse your merchandise , rep ##ack and consolidate it , deliver it to retailers .

reference_raw: We can store your goods, repack and consolidate them, deliver them to the retailers.

reference_spacy_tokens: We can store your goods , repack and consolidate them , deliver them to the retailers .


fn_spacy_tokens_for_baseline: Her hands are claws she keeps concealed within her rags and tatters .

source_bert_toks: her hands are claws she keeps concealed within her rags and ta ##tters .

reference_raw: Her hands are claws that she hides in her rags and shreds.

reference_spacy_tokens: Her hands are claws that she hides in her rags and shreds .


fn_spacy_tokens_for_baseline: He was still there when she showered , dressed and went to partake of what was now her standard breakfast , a flavoursome yoghurt , bread , cheese and coffee .

source_bert_toks: he was still there when she showered , dressed and went to part ##ake of what was now her standard breakfast , a flavour ##some yo ##gh ##urt , bread , cheese and coffee .

reference_raw: He was still there when she showered, dressed, and went to partake of what was now her usual breakfast, tasteful yogurt, bread, cheese, and coffee.

reference_spacy_tokens: He was still there when she showered , dressed , and went to partake of what was now her usual breakfast , tasteful yogurt , bread , cheese , and coffee .


fn_spacy_tokens_for_baseline: The first wells were set alight in February 1991 and their smoke plumes were clearly visible on satellite images of the region .

source_bert_toks: the first wells were set ali ##ght in february 1991 and their smoke plume ##s were clearly visible on satellite images of the region .

reference_raw: The first wells were set on fire in February 1991 and the smoke plumes were clearly visible in the satellite imagery of the region.

reference_spacy_tokens: The first wells were set on fire in February 1991 and the smoke plumes were clearly visible in the satellite imagery of the region .


fn_spacy_tokens_for_baseline: Miss Kitt makes even her gauche opening line -- ' The amount of stars are infinitesimal . ' -- sound sensible and exotically sexy ; from there on she slinks effortlessly through the role , prowling like a panther caged in a pet shop .

source_bert_toks: miss kit ##t makes even her ga ##uche opening line - - ' the amount of stars are infinite ##si ##mal . ' - - sound sensible and exotic ##ally sexy ; from there on she sl ##ink ##s effortlessly through the role , pro ##wl ##ing like a panther cage ##d in a pet shop .

reference_raw: Miss Kitt makes her gauche opening line - 'The number of stars is infinite.' - sounds reasonable and exotically sexy; from there she slips effortlessly through the role and prowls like a panther caged in a pet shop.

reference_spacy_tokens: Miss Kitt makes her gauche opening line - ' The number of stars is infinite . ' - sounds reasonable and exotically sexy ; from there she slips effortlessly through the role and prowls like a panther caged in a pet shop .


fn_spacy_tokens_for_baseline: But soon , yes , she would have to take the telegram and walk to the shop and watch Enid fainting and Mother rustling out from behind the counter .

source_bert_toks: but soon , yes , she would have to take the telegram and walk to the shop and watch en ##id faint ##ing and mother rustling out from behind the counter .

reference_raw: But soon, yes, he would have to take the telegram and go to the store and watch Enid pass out and his mother whisper from behind the counter.

reference_spacy_tokens: But soon , yes , he would have to take the telegram and go to the store and watch Enid pass out and his mother whisper from behind the counter .


fn_spacy_tokens_for_baseline: The true extent of his collections was not discovered until after his death , when the Andy Warhol Museum in Pittsburgh took in 641 boxes of his " Stuff . "

source_bert_toks: the true extent of his collections was not discovered until after his death , when the andy war ##hol museum in pittsburgh took in 64 ##1 boxes of his " stuff . "

reference_raw: The true size of his collection was discovered only after his death, when the Andy Warhol Museum in Pittsburgh took 641 boxes of his stuff.

reference_spacy_tokens: The true size of his collection was discovered only after his death , when the Andy Warhol Museum in Pittsburgh took 641 boxes of his stuff .


fn_spacy_tokens_for_baseline: After all , the buyer wanted to own a car not a computer program .

source_bert_toks: after all , the buyer wanted to own a car not a computer program .

reference_raw: After all, he wanted to own a car, not a computer program.

reference_spacy_tokens: After all , he wanted to own a car , not a computer program .


fn_spacy_tokens_for_baseline: Cartoonists showed the White House half - askew on this basement , noxious odours rising from the grates , while the President snoozed upstairs .

source_bert_toks: cartoonist ##s showed the white house half - ask ##ew on this basement , no ##xious o ##dou ##rs rising from the gr ##ates , while the president s ##no ##oz ##ed upstairs .

reference_raw: Cartoonists showed the White House half-asked in the cellar, the smell of poison rising from the grilles, while the President snooped upstairs.

reference_spacy_tokens: Cartoonists showed the White House half - asked in the cellar , the smell of poison rising from the grilles , while the President snooped upstairs .


fn_spacy_tokens_for_baseline: Turning back to the storm - rinsed windows and the dangerous electrical crackle of Darkfall lightning on the glass , he popped the pill and watched the water running down the glass in ever - increasing and mysterious rivulets .

source_bert_toks: turning back to the storm - ri ##nsed windows and the dangerous electrical crack ##le of dark ##fall lightning on the glass , he popped the pill and watched the water running down the glass in ever - increasing and mysterious ri ##vu ##lets .

reference_raw: He turned back to the stormy windows and the dangerous electric crackle of Darkfall lightning on the glass, took the pill, and watched the water run down the glass in ever-increasing and mysterious streams.

reference_spacy_tokens: He turned back to the stormy windows and the dangerous electric crackle of Darkfall lightning on the glass , took the pill , and watched the water run down the glass in ever - increasing and mysterious streams .


fn_spacy_tokens_for_baseline: ' Most of it was sopped up by the double thickness of the collar at the back but it soaked down through .

source_bert_toks: ' most of it was so ##pped up by the double thickness of the collar at the back but it soaked down through .

reference_raw: Most of it was wet with the double thickness of the collar at the back, but it was wet through.

reference_spacy_tokens: Most of it was wet with the double thickness of the collar at the back , but it was wet through .


fn_spacy_tokens_for_baseline: It is clear that both tobacco and alcohol are risk factors in the development of oral carcinoma .

source_bert_toks: it is clear that both tobacco and alcohol are risk factors in the development of oral car ##cino ##ma .

reference_raw: It is clear that tobacco and alcohol are risk factors for developing oral cancer.

reference_spacy_tokens: It is clear that tobacco and alcohol are risk factors for developing oral cancer .


fn_spacy_tokens_for_baseline: When 26-year - old Tim Haller , from Horncastle , Lincs , bought a double - decker bus in 1988 , his great ambition was to travel around the world in it .

source_bert_toks: when 26 - year - old tim halle ##r , from horn ##castle , lin ##cs , bought a double - decker bus in 1988 , his great ambition was to travel around the world in it .

reference_raw: When, in 1988, 26-year-old Tim Haller, from Horncastle, Lincs, bought a double-decker bus, he wanted to travel around the world in it.

reference_spacy_tokens: When , in 1988 , 26-year - old Tim Haller , from Horncastle , Lincs , bought a double - decker bus , he wanted to travel around the world in it .


fn_spacy_tokens_for_baseline: This had now been soldered to the main building so well that you could n't see the join .

source_bert_toks: this had now been sold ##ered to the main building so well that you couldn ' t see the join .

reference_raw: Now it was sold to the main building so well you couldn't see it.

reference_spacy_tokens: Now it was sold to the main building so well you could n't see it .


fn_spacy_tokens_for_baseline: Paul 's tone and expression showed his puzzlement .

source_bert_toks: paul ' s tone and expression showed his puzzle ##ment .

reference_raw: Paul's voice and expression showed confusion.

reference_spacy_tokens: Paul 's voice and expression showed confusion .


fn_spacy_tokens_for_baseline: No one claimed responsibility for his killing .

source_bert_toks: no one claimed responsibility for his killing .

reference_raw: Nobody claimed responsibility for his murder.

reference_spacy_tokens: Nobody claimed responsibility for his murder .


fn_spacy_tokens_for_baseline: Her time in custody was the equivalent of a one - month prison sentence and she had also had the punishment of spending three weeks at the probation hostel , he added .

source_bert_toks: her time in custody was the equivalent of a one - month prison sentence and she had also had the punishment of spending three weeks at the probation hostel , he added .

reference_raw: Her time in custody was equivalent to one month's imprisonment, and she was sentenced to three weeks' imprisonment, he added.

reference_spacy_tokens: Her time in custody was equivalent to one month 's imprisonment , and she was sentenced to three weeks ' imprisonment , he added .


fn_spacy_tokens_for_baseline: All teachers shall be given a standard competence test to verify they are up - to - date with subjects and modern teaching methods .

source_bert_toks: all teachers shall be given a standard competence test to verify they are up - to - date with subjects and modern teaching methods .

reference_raw: All teachers will receive a standard skill test to check that they are up-to-date with subjects and modern teaching methods.

reference_spacy_tokens: All teachers will receive a standard skill test to check that they are up - to - date with subjects and modern teaching methods .


fn_spacy_tokens_for_baseline: Test blasts shake up residents

source_bert_toks: test blasts shake up residents

reference_raw: Test explosions shock the residents

reference_spacy_tokens: Test explosions shock the residents


fn_spacy_tokens_for_baseline: I sobered , thinking of sagas I had heard of wolves , but my father hushed me .

source_bert_toks: i sober ##ed , thinking of saga ##s i had heard of wolves , but my father hushed me .

reference_raw: I sat up and thought of the sagas I'd heard about the wolves, but my father silenced me.

reference_spacy_tokens: I sat up and thought of the sagas I 'd heard about the wolves , but my father silenced me .


fn_spacy_tokens_for_baseline: Use as a medicine : The repeated episodes of anxiety stimulate secretion of adrenalin and this may be used subconsciously as a stimulant .

source_bert_toks: use as a medicine : the repeated episodes of anxiety stimulate secret ##ion of ad ##ren ##ali ##n and this may be used subconscious ##ly as a st ##im ##ula ##nt .

reference_raw: Use as a medicine: Recurring episodes of anxiety stimulate the secretion of adrenalin, which can be used unconsciously as a stimulant.

reference_spacy_tokens: Use as a medicine : Recurring episodes of anxiety stimulate the secretion of adrenalin , which can be used unconsciously as a stimulant .


fn_spacy_tokens_for_baseline: Leannda , daughter of Courtaulds Fibres employee Chris Ward -see story above - needs expensive eye surgery in America and the Chris Lumb Fund has contributed £ 3,644.47 towards her treatment .

source_bert_toks: lean ##nda , daughter of court ##au ##ld ##s fibre ##s employee chris ward - see story above - needs expensive eye surgery in america and the chris lu ##mb fund has contributed £3 , 64 ##4 . 47 towards her treatment .

reference_raw: Leannda, daughter of the Courtaulds Fibres employee Chris Ward - see the story above - needs expensive eye surgery in America, and Chris Lumb's fund contributed £3,644.47 to her care.

reference_spacy_tokens: Leannda , daughter of the Courtaulds Fibres employee Chris Ward - see the story above - needs expensive eye surgery in America , and Chris Lumb 's fund contributed £ 3,644.47 to her care .


fn_spacy_tokens_for_baseline: Plummer exhaled deeply and gripped the receiver tightly in his hand .

source_bert_toks: plum ##mer exhaled deeply and gripped the receiver tightly in his hand .

reference_raw: Plummer let out a deep breath and gripped the receiver tightly in his hand.

reference_spacy_tokens: Plummer let out a deep breath and gripped the receiver tightly in his hand .


fn_spacy_tokens_for_baseline: She had her hair in two pigtails .

source_bert_toks: she had her hair in two pig ##tails .

reference_raw: Her hair was in two tails.

reference_spacy_tokens: Her hair was in two tails .


fn_spacy_tokens_for_baseline: All the time they stroked their nicotine stained moustaches and exhaled noxious fumes from their foam - flecked lips .

source_bert_toks: all the time they stroked their nico ##tine stained mo ##ust ##ache ##s and exhaled no ##xious fu ##mes from their foam - fl ##eck ##ed lips .

reference_raw: They'd been stroking their nicotine-stained mustaches all the time, breathing noxious fumes from their foamy lips.

reference_spacy_tokens: They 'd been stroking their nicotine - stained mustaches all the time , breathing noxious fumes from their foamy lips .


fn_spacy_tokens_for_baseline: The unique marble hall , which gave its name to the palace , was screened with boards that served as exhibition stands for the photographs , diagrams and models .

source_bert_toks: the unique marble hall , which gave its name to the palace , was screened with boards that served as exhibition stands for the photographs , diagrams and models .

reference_raw: The unique marble hall that had given its name to the palace was covered with boards that served as an exhibition stand for photographs, diagrams, and models.

reference_spacy_tokens: The unique marble hall that had given its name to the palace was covered with boards that served as an exhibition stand for photographs , diagrams , and models .


fn_spacy_tokens_for_baseline: opening hairgrips with their teeth .

source_bert_toks: opening hair ##gr ##ip ##s with their teeth .

reference_raw: open the hairs with their teeth.

reference_spacy_tokens: open the hairs with their teeth .


fn_spacy_tokens_for_baseline: The first time they said that the foetus had detached itself from the placenta , and the second time they said I had a weak cervix because it dilated too early .

source_bert_toks: the first time they said that the foe ##tus had detached itself from the place ##nta , and the second time they said i had a weak ce ##r ##vi ##x because it dil ##ated too early .

reference_raw: The first time they said the fetus was separated from the placenta, and the second time they said I had a weak cervix because it spread too soon.

reference_spacy_tokens: The first time they said the fetus was separated from the placenta , and the second time they said I had a weak cervix because it spread too soon .


fn_spacy_tokens_for_baseline: Two large fires in the New South Wales Blue Mountains , West of Sydney are burning out of control in difficult fire fighting conditions .

source_bert_toks: two large fires in the new south wales blue mountains , west of sydney are burning out of control in difficult fire fighting conditions .

reference_raw: Two big fires in the Blue Mountains of New South Wales, west of Sydney, burn out of control under difficult fire-fighting conditions.

reference_spacy_tokens: Two big fires in the Blue Mountains of New South Wales , west of Sydney , burn out of control under difficult fire - fighting conditions .


fn_spacy_tokens_for_baseline: Nicholson , already a little confused by the claret , was flummoxed .

source_bert_toks: nicholson , already a little confused by the clare ##t , was flu ##mm ##ox ##ed .

reference_raw: Nicholson, already somewhat confused by the claret, was flummoxed.

reference_spacy_tokens: Nicholson , already somewhat confused by the claret , was flummoxed .


fn_spacy_tokens_for_baseline: It is clear that such a strategy would produce an incorrect parsing when a short word followed by a long word is homophonous with a long word followed by a short word .

source_bert_toks: it is clear that such a strategy would produce an incorrect par ##sing when a short word followed by a long word is homo ##phon ##ous with a long word followed by a short word .

reference_raw: It is clear that such a strategy would lead to an incorrect analysis if the short word followed by the long word were homophony with the long word followed by the short word.

reference_spacy_tokens: It is clear that such a strategy would lead to an incorrect analysis if the short word followed by the long word were homophony with the long word followed by the short word .


fn_spacy_tokens_for_baseline: It is more important for them to defer their childbearing and to compress it into a short space of time in their married life .

source_bert_toks: it is more important for them to def ##er their child ##be ##aring and to com ##press it into a short space of time in their married life .

reference_raw: It is more important that they postpone childbearing and compress it in the short term of their marriage.

reference_spacy_tokens: It is more important that they postpone childbearing and compress it in the short term of their marriage .


fn_spacy_tokens_for_baseline: In the metropolis one would experience much bigger crowds , more atmosphere , louder roars and at times very malodorous sawdust .

source_bert_toks: in the metropolis one would experience much bigger crowds , more atmosphere , louder roar ##s and at times very mal ##od ##orous saw ##dust .

reference_raw: There were much larger crowds in the metropolis, more atmosphere, louder roars, and sometimes very bad sawdust.

reference_spacy_tokens: There were much larger crowds in the metropolis , more atmosphere , louder roars , and sometimes very bad sawdust .


fn_spacy_tokens_for_baseline: Slamming it after him , a look of disgust on her face , Hilary poured out her tea then carried the cup to the table and sat nursing it in her palms .

source_bert_toks: slamming it after him , a look of disgust on her face , hilary poured out her tea then carried the cup to the table and sat nursing it in her palms .

reference_raw: Hilary slapped her behind him, a look of disgust on her face, poured out her tea, then carried the cup to the table and sat nursing it in her hands.

reference_spacy_tokens: Hilary slapped her behind him , a look of disgust on her face , poured out her tea , then carried the cup to the table and sat nursing it in her hands .


fn_spacy_tokens_for_baseline: But the most interesting thing to do here is to explore the two villages on the island , Cheung Chau and San Wai .

source_bert_toks: but the most interesting thing to do here is to explore the two villages on the island , cheung cha ##u and san wai .

reference_raw: But most interestingly, we're looking at the two villages on the island, Cheung Chau and San Wai.

reference_spacy_tokens: But most interestingly , we 're looking at the two villages on the island , Cheung Chau and San Wai .


fn_spacy_tokens_for_baseline: Such planning is crucial if our hopes for community care are to be realised after 1993 .

source_bert_toks: such planning is crucial if our hopes for community care are to be realised after 1993 .

reference_raw: Such planning is essential if our expectations for community care are to be realised after 1993.

reference_spacy_tokens: Such planning is essential if our expectations for community care are to be realised after 1993 .


fn_spacy_tokens_for_baseline: She was badly injured about the head and many other bones in her body were smashed .

source_bert_toks: she was badly injured about the head and many other bones in her body were smashed .

reference_raw: She had a severe head injury, and a lot of other bones in her body had been crushed.

reference_spacy_tokens: She had a severe head injury , and a lot of other bones in her body had been crushed .


fn_spacy_tokens_for_baseline: The view is especially effective at night .

source_bert_toks: the view is especially effective at night .

reference_raw: The view is particularly efficient at night.

reference_spacy_tokens: The view is particularly efficient at night .


fn_spacy_tokens_for_baseline: Thereafter , Yeremi himself would never again be fazed by being decanted into any foreign environment .

source_bert_toks: thereafter , yer ##emi himself would never again be fa ##zed by being dec ##ante ##d into any foreign environment .

reference_raw: After that, Yeremi himself would never again be fascinated by being decanted in a foreign environment.

reference_spacy_tokens: After that , Yeremi himself would never again be fascinated by being decanted in a foreign environment .


fn_spacy_tokens_for_baseline: She clumped across the star - patterned mosaic towards the walk - in cupboard where the ski - gear was kept .

source_bert_toks: she cl ##ump ##ed across the star - patterned mosaic towards the walk - in cupboard where the ski - gear was kept .

reference_raw: She crouched over the star-pattered mosaic to the walk-in closet where she kept her ski gear.

reference_spacy_tokens: She crouched over the star - pattered mosaic to the walk - in closet where she kept her ski gear .


fn_spacy_tokens_for_baseline: She turned towards the pantry and slipped her fingers beneath the slats .

source_bert_toks: she turned towards the pantry and slipped her fingers beneath the sl ##ats .

reference_raw: She turned to the cupboard and slipped her fingers under the slats.

reference_spacy_tokens: She turned to the cupboard and slipped her fingers under the slats .


fn_spacy_tokens_for_baseline: Two days later he actually came , an oldish man , at least sixty , and valued two of the cabinets as worth £ 500 apiece .

source_bert_toks: two days later he actually came , an old ##ish man , at least sixty , and valued two of the cabinets as worth £5 ##00 api ##ece .

reference_raw: In fact, he arrived two days later, an old man of at least sixty, and valued two cupboards for five hundred pounds each.

reference_spacy_tokens: In fact , he arrived two days later , an old man of at least sixty , and valued two cupboards for five hundred pounds each .


fn_spacy_tokens_for_baseline: Searches also uncover small stashes of ammonium nitrate fertilizer , the main bomb - making ingredient .

source_bert_toks: searches also uncover small st ##ash ##es of am ##monium nitrate fe ##rti ##lizer , the main bomb - making ingredient .

reference_raw: The search also reveals small pockets of ammonium nitrate fertilizer, a major ingredient in the manufacture of bombs.

reference_spacy_tokens: The search also reveals small pockets of ammonium nitrate fertilizer , a major ingredient in the manufacture of bombs .


fn_spacy_tokens_for_baseline: The program was revived later in the decade , when strategic interests began to drive the nuclear program .

source_bert_toks: the program was revived later in the decade , when strategic interests began to drive the nuclear program .

reference_raw: The program was revived late in the decade, when strategic interests began to drive the nuclear program.

reference_spacy_tokens: The program was revived late in the decade , when strategic interests began to drive the nuclear program .


fn_spacy_tokens_for_baseline: The local promoter had evidently not managed to summon the courage to quibble with Jourgensen over the 89 decibel limit on the main stage .

source_bert_toks: the local promoter had evidently not managed to summon the courage to qui ##bble with jo ##urg ##ense ##n over the 89 dec ##ibe ##l limit on the main stage .

reference_raw: The local promoter didn't seem to have the courage to argue with Jourgensen over the 89 decibel limit on the main stage.

reference_spacy_tokens: The local promoter did n't seem to have the courage to argue with Jourgensen over the 89 decibel limit on the main stage .


fn_spacy_tokens_for_baseline: Shenzhen is easy to reach - the KCR commuter train runs throughout the day , the trip taking about 40 minutes .

source_bert_toks: shenzhen is easy to reach - the kc ##r commuter train runs throughout the day , the trip taking about 40 minutes .

reference_raw: It's easy to reach Shenzhen - the KCR commuter train operates all day, and the journey takes about 40 minutes.

reference_spacy_tokens: It 's easy to reach Shenzhen - the KCR commuter train operates all day , and the journey takes about 40 minutes .


fn_spacy_tokens_for_baseline: His certainty , his sense of direction , his belief in himself disintegrated alongside his career , and the Nizan household was never the same again .

source_bert_toks: his certainty , his sense of direction , his belief in himself di ##sin ##te ##grate ##d alongside his career , and the ni ##zan household was never the same again .

reference_raw: His confidence, his sense of direction, his faith in himself had fallen apart with his career, and the Nizan family had never been the same.

reference_spacy_tokens: His confidence , his sense of direction , his faith in himself had fallen apart with his career , and the Nizan family had never been the same .


fn_spacy_tokens_for_baseline: It will help policemen and insurance assessors to extract sensible evidence from befuddled witnesses , tease effective instructions out of skilled but inarticulate craftsmen , and trick unwary criminals into much completer confessions than they had intended .

source_bert_toks: it will help policemen and insurance assess ##ors to extract sensible evidence from be ##fu ##ddled witnesses , tease effective instructions out of skilled but ina ##rti ##cula ##te craftsmen , and trick un ##war ##y criminals into much complete ##r confessions than they had intended .

reference_raw: It will help police and insurance assessors to extract reasonable evidence from confused witnesses, tease effective instructions from experienced but inarticulate craftsmen, and deceive unwary criminals into far more complete confessions than they intended.

reference_spacy_tokens: It will help police and insurance assessors to extract reasonable evidence from confused witnesses , tease effective instructions from experienced but inarticulate craftsmen , and deceive unwary criminals into far more complete confessions than they intended .


fn_spacy_tokens_for_baseline: The survey appears to contradict motor industry claims that customers still rate high performance as one of the most desirable attributes .

source_bert_toks: the survey appears to contra ##dict motor industry claims that customers still rate high performance as one of the most desirable attributes .

reference_raw: The survey seems to be in contradiction with the claims of the motor industry that customers continue to assess high performance as one of the most desirable characteristics.

reference_spacy_tokens: The survey seems to be in contradiction with the claims of the motor industry that customers continue to assess high performance as one of the most desirable characteristics .


fn_spacy_tokens_for_baseline: The interception of their boat was part of a growing pattern of interception of illegal would - be immigrants into Spain .

source_bert_toks: the interception of their boat was part of a growing pattern of interception of illegal would - be immigrants into spain .

reference_raw: The capture of their ship was part of a growing pattern of intercepting illegal, would-be immigrants from Spain.

reference_spacy_tokens: The capture of their ship was part of a growing pattern of intercepting illegal , would - be immigrants from Spain .


fn_spacy_tokens_for_baseline: LSD , mescalin , ' ' magic mushrooms ' , ecstasy and similar ' designer ' ' drugs that are synthetically manufactured hallucinogens . v amphetamines , amyl nitrite and other stimulants ( ' uppers ' ) .

source_bert_toks: l ##sd , me ##sca ##lin , ' ' magic mushrooms ' , ecstasy and similar ' designer ' ' drugs that are synthetic ##ally manufactured hall ##uc ##ino ##gens . v amp ##het ##amine ##s , amy ##l ni ##tri ##te and other st ##im ##ula ##nts ( ' upper ##s ' ) .

reference_raw: LSD, mescaline, 'magic mushrooms' , ecstasy and the like 'designer' 'drugs which are synthetic hallucinogens. v amphetamines, amyl nitrite and other stimulants ('uppers' ).

reference_spacy_tokens: LSD , mescaline , ' magic mushrooms ' , ecstasy and the like ' designer ' ' drugs which are synthetic hallucinogens . v amphetamines , amyl nitrite and other stimulants ( ' uppers ' ) .


fn_spacy_tokens_for_baseline: The Duke and Duchess of Gloucester will honour the ' Stars of the Bolshoi ' opening night a week tomorrow at the Dominion Theatre in London .

source_bert_toks: the duke and duchess of gloucester will honour the ' stars of the bo ##ls ##ho ##i ' opening night a week tomorrow at the dominion theatre in london .

reference_raw: The Duke and Duke of Gloucester will honor the opening night of the Bolshoi Stars tomorrow at the Dominion Theatre in London.

reference_spacy_tokens: The Duke and Duke of Gloucester will honor the opening night of the Bolshoi Stars tomorrow at the Dominion Theatre in London .


fn_spacy_tokens_for_baseline: Some sports magazines and shops sell weighted strips of material that you can wear around your wrists or ankles whilst doing other exercise ( such as suppleness exercises or when running ) .

source_bert_toks: some sports magazines and shops sell weighted strips of material that you can wear around your wrists or ankles whilst doing other exercise ( such as su ##pp ##lene ##ss exercises or when running ) .

reference_raw: Some sporting magazines and shops sell weighted strips of material that can be worn around the wrists or knuckles while performing other exercises (e.g. exercises or running).

reference_spacy_tokens: Some sporting magazines and shops sell weighted strips of material that can be worn around the wrists or knuckles while performing other exercises ( e.g. exercises or running ) .


fn_spacy_tokens_for_baseline: She was butchered at the wheel of the vehicle by her attacker who hung through the passenger window , repeatedly slicing her with the panga .

source_bert_toks: she was butcher ##ed at the wheel of the vehicle by her attacker who hung through the passenger window , repeatedly slicing her with the pang ##a .

reference_raw: At the wheel of the car, she was massacred by an assailant who hung in the passenger window and repeatedly cut her with a panga.

reference_spacy_tokens: At the wheel of the car , she was massacred by an assailant who hung in the passenger window and repeatedly cut her with a panga .


fn_spacy_tokens_for_baseline: Charles unbuckled his ski straps and stuck skis and sticks in the snow beside Peregrine 's .

source_bert_toks: charles un ##buck ##led his ski straps and stuck ski ##s and sticks in the snow beside pere ##grin ##e ' s .

reference_raw: Charles unbuttoned his ski straps and stuck the skis and sticks in the snow next to Peregrine's.

reference_spacy_tokens: Charles unbuttoned his ski straps and stuck the skis and sticks in the snow next to Peregrine 's .


fn_spacy_tokens_for_baseline: He made himself as meek as a lamb , hoping perhaps that they would give him an extra morsel of food or some other favour .

source_bert_toks: he made himself as meek as a lamb , hoping perhaps that they would give him an extra morse ##l of food or some other favour .

reference_raw: He made himself look like a sheep, hoping they might give him an extra bite of food or some other favor.

reference_spacy_tokens: He made himself look like a sheep , hoping they might give him an extra bite of food or some other favor .


fn_spacy_tokens_for_baseline: Harry Symmonds gave evidence that the carpet bag holding one of the tiny bodies found in Clapper 's Pool was the one he had brought to Mrs Dyer 's house .

source_bert_toks: harry sy ##mmon ##ds gave evidence that the carpet bag holding one of the tiny bodies found in clap ##per ' s pool was the one he had brought to mrs dyer ' s house .

reference_raw: Harry Symmonds testified that the carpet bag containing one of the small bodies found in Clapper's pool was the one he had taken to Mrs. Dyer's house.

reference_spacy_tokens: Harry Symmonds testified that the carpet bag containing one of the small bodies found in Clapper 's pool was the one he had taken to Mrs. Dyer 's house .


fn_spacy_tokens_for_baseline: According to defector accounts , the DPRK 's long - range missiles such as the Nodong , and other ballistic rockets and artillery pieces with calibres larger than 80 mm , are capable of delivering CW agents .

source_bert_toks: according to defect ##or accounts , the d ##pr ##k ' s long - range missiles such as the nod ##ong , and other ballistic rockets and artillery pieces with cal ##ib ##res larger than 80 mm , are capable of delivering cw agents .

reference_raw: According to defector accounts, DPRK's long-range missiles, like the Nodong, and other ballistic missiles and cannons with a calibre greater than 80 mm, are capable of delivering CW agents.

reference_spacy_tokens: According to defector accounts , DPRK 's long - range missiles , like the Nodong , and other ballistic missiles and cannons with a calibre greater than 80 mm , are capable of delivering CW agents .


fn_spacy_tokens_for_baseline: When a pregnant woman smokes she inhales various poisons , including nicotine and carbon monoxide , which are absorbed into her body and pass to the baby .

source_bert_toks: when a pregnant woman smoke ##s she in ##hale ##s various poison ##s , including nico ##tine and carbon mono ##xide , which are absorbed into her body and pass to the baby .

reference_raw: When a pregnant woman smokes, she breathes a variety of poisons, including nicotine and carbon monoxide, that are absorbed into her body and passed on to the child.

reference_spacy_tokens: When a pregnant woman smokes , she breathes a variety of poisons , including nicotine and carbon monoxide , that are absorbed into her body and passed on to the child .


fn_spacy_tokens_for_baseline: In a feature interview with Percy Grainger next day I found him to be a charming and modest man , and was glad to write in glowing terms of his keyboard mastery , his repertoire , and his travels .

source_bert_toks: in a feature interview with percy grain ##ger next day i found him to be a charming and modest man , and was glad to write in glowing terms of his keyboard mastery , his repertoire , and his travels .

reference_raw: The next day, in a typical interview with Percy Grainger, I found him a charming and modest man, and I was happy to write in shining terms about his keyboard control, his repertoire, and his travels.

reference_spacy_tokens: The next day , in a typical interview with Percy Grainger , I found him a charming and modest man , and I was happy to write in shining terms about his keyboard control , his repertoire , and his travels .


fn_spacy_tokens_for_baseline: Electronic theft by foreign and industrial spies and disgruntled employees is costing U.S. companies billions and eroding their international competitive advantage .

source_bert_toks: electronic theft by foreign and industrial spies and di ##sg ##run ##tled employees is costing u . s . companies billions and er ##od ##ing their international competitive advantage .

reference_raw: The electronic theft of foreign and industrial spies and disgruntled employees is costing American companies billions of dollars and eroding their international competitiveness advantage.

reference_spacy_tokens: The electronic theft of foreign and industrial spies and disgruntled employees is costing American companies billions of dollars and eroding their international competitiveness advantage .


fn_spacy_tokens_for_baseline: The fire fighting is focused on areas where the fire threatens buildings .

source_bert_toks: the fire fighting is focused on areas where the fire threatens buildings .

reference_raw: Firefighting focuses on areas where fire endangers buildings.

reference_spacy_tokens: Firefighting focuses on areas where fire endangers buildings .


fn_spacy_tokens_for_baseline: You can prove this for yourself , in fact , by re - reading any good whodunit where you can remember who the murder eventually proved to be .

source_bert_toks: you can prove this for yourself , in fact , by re - reading any good who ##dun ##it where you can remember who the murder eventually proved to be .

reference_raw: In fact, you can prove it yourself by rereading all the good whodunits where you remember who the murder turned out to be.

reference_spacy_tokens: In fact , you can prove it yourself by rereading all the good whodunits where you remember who the murder turned out to be .


fn_spacy_tokens_for_baseline: According to the statement , the first of its kind , the group has managed to manufacture and develop at least 20 different types of biological and chemical weapons .

source_bert_toks: according to the statement , the first of its kind , the group has managed to manufacture and develop at least 20 different types of biological and chemical weapons .

reference_raw: According to that statement, the first of its kind, the group was able to produce and develop at least 20 different types of biological and chemical weapons.

reference_spacy_tokens: According to that statement , the first of its kind , the group was able to produce and develop at least 20 different types of biological and chemical weapons .


fn_spacy_tokens_for_baseline: The grain is first milled , or ground to a meal .

source_bert_toks: the grain is first mill ##ed , or ground to a meal .

reference_raw: The grain is first ground or ground to food.

reference_spacy_tokens: The grain is first ground or ground to food .


fn_spacy_tokens_for_baseline: The brilliance of that moment was underlined by the fact that Williams , playing a man with a particularly sweet tooth , was eating a sour grapefruit without sugar at the time .

source_bert_toks: the brilliance of that moment was under ##lined by the fact that williams , playing a man with a particularly sweet tooth , was eating a sour grape ##fr ##uit without sugar at the time .

reference_raw: The brightness of the moment was underlined by the fact that Williams, who had been playing a man with a particularly sweet tooth, had eaten a bitter grapefruit without sugar.

reference_spacy_tokens: The brightness of the moment was underlined by the fact that Williams , who had been playing a man with a particularly sweet tooth , had eaten a bitter grapefruit without sugar .


fn_spacy_tokens_for_baseline: Clean , scale and gut the sardines and remove the heads .

source_bert_toks: clean , scale and gut the sar ##dine ##s and remove the heads .

reference_raw: Clean, scale, and cut off the sardines and take off the heads.

reference_spacy_tokens: Clean , scale , and cut off the sardines and take off the heads .


fn_spacy_tokens_for_baseline: Murder was committed in order to silence people , and out of jealousy and revenge and anger , and simply for its own sake .

source_bert_toks: murder was committed in order to silence people , and out of jealousy and revenge and anger , and simply for its own sake .

reference_raw: Murder was committed to keep people quiet, out of jealousy and vengeance and anger, and only for his own good.

reference_spacy_tokens: Murder was committed to keep people quiet , out of jealousy and vengeance and anger , and only for his own good .


fn_spacy_tokens_for_baseline: Suicide bombing outside Afghanistan hospital kills 27

source_bert_toks: suicide bombing outside afghanistan hospital kills 27

reference_raw: Suicide-bombing in front of Afghanistan Hospital kills 27

reference_spacy_tokens: Suicide - bombing in front of Afghanistan Hospital kills 27


fn_spacy_tokens_for_baseline: For one thing your own troops will probably end up blocking the line of sight .

source_bert_toks: for one thing your own troops will probably end up blocking the line of sight .

reference_raw: First of all, your own soldiers are likely to end up shutting down the line of sight.

reference_spacy_tokens: First of all , your own soldiers are likely to end up shutting down the line of sight .


fn_spacy_tokens_for_baseline: A young herring gull taps its mother 's beak to trigger her into regurgitating its next meal .

source_bert_toks: a young herring gu ##ll taps its mother ' s beak to trigger her into reg ##urg ##itating its next meal .

reference_raw: A young herring seagull taps her mother's beak to trigger her to regurgitate her next meal.

reference_spacy_tokens: A young herring seagull taps her mother 's beak to trigger her to regurgitate her next meal .


fn_spacy_tokens_for_baseline: TWO kidnapped men were rescued when motorway police stopped a getaway van for going too slowly .

source_bert_toks: two kidnapped men were rescued when motorway police stopped a get ##away van for going too slowly .

reference_raw: The two abducted men were rescued when the highway police stopped the escape van because it was too slow.

reference_spacy_tokens: The two abducted men were rescued when the highway police stopped the escape van because it was too slow .


fn_spacy_tokens_for_baseline: The kids were mortified but I thought it was a real hoot . '

source_bert_toks: the kids were mort ##ified but i thought it was a real ho ##ot . '

reference_raw: The children were humiliated, but I thought it was a real hoot.'

reference_spacy_tokens: The children were humiliated , but I thought it was a real hoot . '


fn_spacy_tokens_for_baseline: He stayed there , retching , leaning to his side on one arm , for several minutes .

source_bert_toks: he stayed there , re ##tch ##ing , leaning to his side on one arm , for several minutes .

reference_raw: He stayed there, twitching, leaning on one arm for a few minutes.

reference_spacy_tokens: He stayed there , twitching , leaning on one arm for a few minutes .


fn_spacy_tokens_for_baseline: And in this city , where literature and theater have historically dominated the scene , visual arts are finally coming into their own with the new Museum of Modern Art and the many galleries that display the work of modern Irish artists .

source_bert_toks: and in this city , where literature and theater have historically dominated the scene , visual arts are finally coming into their own with the new museum of modern art and the many galleries that display the work of modern irish artists .

reference_raw: And in this city, where literature and theatre have historically dominated the scene, the visual arts have finally come to theirs with a new museum of modern art and many galleries that show the work of modern Irish painters.

reference_spacy_tokens: And in this city , where literature and theatre have historically dominated the scene , the visual arts have finally come to theirs with a new museum of modern art and many galleries that show the work of modern Irish painters .


fn_spacy_tokens_for_baseline: I pulled the little guy from my pocket and set him down on my desk .

source_bert_toks: i pulled the little guy from my pocket and set him down on my desk .

reference_raw: I took the little man out of my pocket and put him on the table.

reference_spacy_tokens: I took the little man out of my pocket and put him on the table .


fn_spacy_tokens_for_baseline: Much of the best Islamic architecture in the city was constructed in the Mameluke era , but the past thousand years had taken their toll : Jerusalem was unable to regain the prosperity it had enjoyed in earlier times .

source_bert_toks: much of the best islamic architecture in the city was constructed in the ma ##mel ##uke era , but the past thousand years had taken their toll : jerusalem was unable to regain the prosperity it had enjoyed in earlier times .

reference_raw: Most of the best Islamic architecture in the city had been built in the Mameluke era, but the last thousand years had taken their toll: Jerusalem could not regain the prosperity it had enjoyed in the past.

reference_spacy_tokens: Most of the best Islamic architecture in the city had been built in the Mameluke era , but the last thousand years had taken their toll : Jerusalem could not regain the prosperity it had enjoyed in the past .


fn_spacy_tokens_for_baseline: He is answered by Joseph who proudly unzips a handsomely bound , black leather , loose - leaf portfolio featuring photos ( taken by his friends on the scheme and selected , developed and printed by Joseph ) , text and cartoons all illustrating Joseph working , playing and learning .

source_bert_toks: he is answered by joseph who proudly un ##zi ##ps a handsome ##ly bound , black leather , loose - leaf portfolio featuring photos ( taken by his friends on the scheme and selected , developed and printed by joseph ) , text and cartoons all illustrating joseph working , playing and learning .

reference_raw: He is answered by Joseph, who proudly unzips the handsomely bound, black leather, loose-leaved portfolio photographs (taken by his friends on the plan and selected, produced and printed by Joseph), text and cartoons illustrating Joseph's work, play and study.

reference_spacy_tokens: He is answered by Joseph , who proudly unzips the handsomely bound , black leather , loose - leaved portfolio photographs ( taken by his friends on the plan and selected , produced and printed by Joseph ) , text and cartoons illustrating Joseph 's work , play and study .


fn_spacy_tokens_for_baseline: Iran responded by strenuously denying the charges , but also maintained that it reserved the right to possess nuclear weapons to counter Israel 's weaponry .

source_bert_toks: iran responded by st ##ren ##uously denying the charges , but also maintained that it reserved the right to possess nuclear weapons to counter israel ' s weaponry .

reference_raw: Iran responded by strenuously denying the charges, but also asserting that it reserved the right to own nuclear weapons against Israeli weapons.

reference_spacy_tokens: Iran responded by strenuously denying the charges , but also asserting that it reserved the right to own nuclear weapons against Israeli weapons .


fn_spacy_tokens_for_baseline: Both of Jaq 's parents were adepts of genetics .

source_bert_toks: both of ja ##q ' s parents were adept ##s of genetics .

reference_raw: Both Jaq's parents were genetics experts.

reference_spacy_tokens: Both Jaq 's parents were genetics experts .


fn_spacy_tokens_for_baseline: The less compatible a pair the greater the probability that their relationship will regress in the presence of another animal .

source_bert_toks: the less compatible a pair the greater the probability that their relationship will reg ##ress in the presence of another animal .

reference_raw: The less compatible a couple, the more likely their relationship is to regress in the presence of another animal.

reference_spacy_tokens: The less compatible a couple , the more likely their relationship is to regress in the presence of another animal .


fn_spacy_tokens_for_baseline: Branches of Dillons , Books Etc and Bookland have been taken to task over the wording on their sale signposting and promotional material , including material provided by the National Book Sale .

source_bert_toks: branches of dillon ##s , books etc and book ##land have been taken to task over the word ##ing on their sale sign ##post ##ing and promotional material , including material provided by the national book sale .

reference_raw: Branches of Dillons, Books Etc and Bookland were assigned to discuss the wording of their sales labels and promotional material, including the material provided by the National Book Sale.

reference_spacy_tokens: Branches of Dillons , Books Etc and Bookland were assigned to discuss the wording of their sales labels and promotional material , including the material provided by the National Book Sale .


fn_spacy_tokens_for_baseline: He paused to throw another fierce glance in their direction .

source_bert_toks: he paused to throw another fierce glance in their direction .

reference_raw: He stopped and gave them another wild look.

reference_spacy_tokens: He stopped and gave them another wild look .


fn_spacy_tokens_for_baseline: When he lost to the now - tragic figure of Michael Watson in 1989 , Benn believed he was invincible -- incredibly he spent nearly five hours on fight - day having his hair plaited .

source_bert_toks: when he lost to the now - tragic figure of michael watson in 1989 , ben ##n believed he was invincible - - incredibly he spent nearly five hours on fight - day having his hair pl ##ai ##ted .

reference_raw: When Benn lost himself to Michael Watson's now-tragic figure in 1989, he believed he was invincible-unbelievably spent almost five hours in combat-day with his hair plaid.

reference_spacy_tokens: When Benn lost himself to Michael Watson 's now - tragic figure in 1989 , he believed he was invincible - unbelievably spent almost five hours in combat - day with his hair plaid .


fn_spacy_tokens_for_baseline: He had gone to one of the windows which stretched all the way to the ceiling and pulled back the lace curtain covering the unwashed panes of glass .

source_bert_toks: he had gone to one of the windows which stretched all the way to the ceiling and pulled back the lace curtain covering the un ##washed pan ##es of glass .

reference_raw: He went to one of the windows that stretched all the way up to the ceiling and pulled back the lace curtain over the unwashed glass panes.

reference_spacy_tokens: He went to one of the windows that stretched all the way up to the ceiling and pulled back the lace curtain over the unwashed glass panes .


fn_spacy_tokens_for_baseline: Boss McDonough quits playing role

source_bert_toks: boss mc ##don ##ough quit ##s playing role

reference_raw: Chief McDonough has stopped playing the role


fn_spacy_tokens_for_baseline: The sound is equal parts glam , punk , and metal , and features Cheap Trick guitarist Rick Nielsen .

source_bert_toks: the sound is equal parts g ##lam , punk , and metal , and features cheap trick guitarist rick nielsen .

reference_raw: Acoustics are equal parts of glam, punk, and metal, and features of Cheap Trick guitarist Rick Nielsen.

reference_spacy_tokens: Acoustics are equal parts of glam , punk , and metal , and features of Cheap Trick guitarist Rick Nielsen .


fn_spacy_tokens_for_baseline: Pretty caramel shapes decorate this Lemon Ginger Cheesecake

source_bert_toks: pretty cara ##mel shapes decorate this lemon ginger cheese ##cake

reference_raw: Nice caramel shapes decorate this lemon ginger cheese pie


fn_spacy_tokens_for_baseline: U. S. military commanders have pressed troops to avoid civilian casualties that could undercut public support for the military efforts and the Afghan government .

source_bert_toks: u . s . military commanders have pressed troops to avoid civilian casualties that could under ##cut public support for the military efforts and the afghan government .

reference_raw: U.S. military commanders urged troops to avoid civilian casualties that could undermine public support for military efforts and the Afghan government.

reference_spacy_tokens: U.S. military commanders urged troops to avoid civilian casualties that could undermine public support for military efforts and the Afghan government .


fn_spacy_tokens_for_baseline: Returning to the living room he killed the light and the images of the previous night , drew the curtains , and threw open the windows and shutters .

source_bert_toks: returning to the living room he killed the light and the images of the previous night , drew the curtains , and threw open the windows and shutter ##s .

reference_raw: He went back into the living room, shutting off the light and images of the previous night, pulled back the curtains, and opened the windows and shutters.

reference_spacy_tokens: He went back into the living room , shutting off the light and images of the previous night , pulled back the curtains , and opened the windows and shutters .


fn_spacy_tokens_for_baseline: In the Atlantic its average breadth is nearly 100 miles , and many of its peaks tower 10,000 feet above the seabed .

source_bert_toks: in the atlantic its average breadth is nearly 100 miles , and many of its peaks tower 10 , 000 feet above the sea ##bed .

reference_raw: The average width of the Atlantic is almost 100 miles, and many of its peaks are ten thousand feet above the seabed.

reference_spacy_tokens: The average width of the Atlantic is almost 100 miles , and many of its peaks are ten thousand feet above the seabed .


fn_spacy_tokens_for_baseline: Chen Jia Ci , the Chen Family Institute ( open daily 8:30 am - 5 pm ) , was built in the late 19th century to promote arts and crafts .

source_bert_toks: chen jia ci , the chen family institute ( open daily 8 : 30 am - 5 pm ) , was built in the late 19th century to promote arts and crafts .

reference_raw: Chen Jia Ci, the Chen Family Institute (open daily at 8:30 - 5 p.m.), was built at the end of the 19th century to promote art and art.

reference_spacy_tokens: Chen Jia Ci , the Chen Family Institute ( open daily at 8:30 - 5 p.m. ) , was built at the end of the 19th century to promote art and art .


fn_spacy_tokens_for_baseline: He becomes strong on channel 2 from " any way you like " , at just before minute 1 .

source_bert_toks: he becomes strong on channel 2 from " any way you like " , at just before minute 1 .

reference_raw: It's going to be powerful on channel 2 from "whatever you want", just before minute one.

reference_spacy_tokens: It 's going to be powerful on channel 2 from " whatever you want " , just before minute one .


fn_spacy_tokens_for_baseline: You can take a ferryboat through the harbor , past the ingenious Plover Cove reservoir , a water catchment area appropriated by damming and draining a broad inlet from the sea .

source_bert_toks: you can take a ferry ##boat through the harbor , past the ing ##eni ##ous pl ##over cove reservoir , a water catchment area appropriated by dam ##ming and draining a broad inlet from the sea .

reference_raw: You can take a ferry ship through the port, through the ingenious pool of Plover Cove, a water catchment area suitable for damming and draining a wide inlet from the sea.

reference_spacy_tokens: You can take a ferry ship through the port , through the ingenious pool of Plover Cove , a water catchment area suitable for damming and draining a wide inlet from the sea .


fn_spacy_tokens_for_baseline: Such transfers have resulted in numerous U.S. nonproliferation sanctions being imposed on Chinese entities .

source_bert_toks: such transfers have resulted in numerous u . s . non ##pro ##life ##ration sanctions being imposed on chinese entities .

reference_raw: These transfers led to the imposition of a number of US non-proliferation sanctions on Chinese operators.

reference_spacy_tokens: These transfers led to the imposition of a number of US non - proliferation sanctions on Chinese operators .


fn_spacy_tokens_for_baseline: But he forbore to comment .

source_bert_toks: but he for ##bor ##e to comment .

reference_raw: But he refrained from commenting.

reference_spacy_tokens: But he refrained from commenting .


fn_spacy_tokens_for_baseline: I winked at my self , jiggled my head about , grinned .

source_bert_toks: i winked at my self , ji ##ggle ##d my head about , grinned .

reference_raw: I blinked at myself, shook my head, and smiled.

reference_spacy_tokens: I blinked at myself , shook my head , and smiled .


fn_spacy_tokens_for_baseline: This meeting has at least concluded with a semblance of good will .

source_bert_toks: this meeting has at least concluded with a se ##mb ##lance of good will .

reference_raw: This meeting, at least, came to an end with a sense of good will.

reference_spacy_tokens: This meeting , at least , came to an end with a sense of good will .


fn_spacy_tokens_for_baseline: A decent blouse and skirt , or an attractive frock , either outfit worn with a clean straw boater , and she ought to be able to get a job in a store .

source_bert_toks: a decent blouse and skirt , or an attractive fr ##ock , either outfit worn with a clean straw boat ##er , and she ought to be able to get a job in a store .

reference_raw: A decent blouse and skirt, or an attractive dress, either wearing a clean straw boat, and she should be able to get a job in a shop.

reference_spacy_tokens: A decent blouse and skirt , or an attractive dress , either wearing a clean straw boat , and she should be able to get a job in a shop .


fn_spacy_tokens_for_baseline: Unlike Diana who was christened in Sandringham church and had well - to - do commoners for godparents , baby brother Charles was christened in style at Westminster Abbey with the Queen as principal godparent .

source_bert_toks: unlike diana who was christ ##ened in sand ##ring ##ham church and had well - to - do common ##ers for god ##par ##ents , baby brother charles was christ ##ened in style at westminster abbey with the queen as principal god ##par ##ent .

reference_raw: Unlike Diana, who had been baptized in Sandringham Church and had well-established commoners for the godfathers, little brother Charles had been baptized in style in Westminster Abbey with the Queen as the principal godfather.

reference_spacy_tokens: Unlike Diana , who had been baptized in Sandringham Church and had well - established commoners for the godfathers , little brother Charles had been baptized in style in Westminster Abbey with the Queen as the principal godfather .


fn_spacy_tokens_for_baseline: Following the raid a number of Hyundai executives were arrested , bringing the total of UPP campaign workers detained for violations of electoral law to at least 70 .

source_bert_toks: following the raid a number of hyundai executives were arrested , bringing the total of up ##p campaign workers detained for violations of electoral law to at least 70 .

reference_raw: After the raid, several Hyundai executives were arrested, bringing the total number of UPP campaign staff arrested for violations of electoral law to at least 70.

reference_spacy_tokens: After the raid , several Hyundai executives were arrested , bringing the total number of UPP campaign staff arrested for violations of electoral law to at least 70 .


fn_spacy_tokens_for_baseline: The reason for distinguishing between murder and manslaughter must be to identify and to label the most heinous killings as murder , and it has been questioned whether English law succeeds in this .

source_bert_toks: the reason for distinguishing between murder and mans ##lau ##ght ##er must be to identify and to label the most he ##ino ##us killings as murder , and it has been questioned whether english law succeeds in this .

reference_raw: The reason for distinguishing between murder and murder must be to identify and identify the most gruesome killings as murder, and the question is whether English law will succeed.

reference_spacy_tokens: The reason for distinguishing between murder and murder must be to identify and identify the most gruesome killings as murder , and the question is whether English law will succeed .


fn_spacy_tokens_for_baseline: A locomotive was gliding in , hauling a trainload of coaches .

source_bert_toks: a locomotive was gliding in , hauling a train ##load of coaches .

reference_raw: A locomotive glided in, hauling a train full of carriages.

reference_spacy_tokens: A locomotive glided in , hauling a train full of carriages .


fn_spacy_tokens_for_baseline: The DHS report describes the three methods currently used by customs officials to inspect the trucks : a Radiation Portal Monitor to detect the presence of radiation , a Vehicle and Cargo Inspection System screening that creates an image similar to an X - ray , and physical inspections .

source_bert_toks: the dh ##s report describes the three methods currently used by customs officials to inspect the trucks : a radiation portal monitor to detect the presence of radiation , a vehicle and cargo inspection system screening that creates an image similar to an x - ray , and physical inspections .

reference_raw: The DHS report describes the three methods currently used by customs officials for the inspection of vehicles: a radiation portal monitor for the detection of radiation, a vehicle and cargo inspection system that creates an image similar to an X-ray, and a physical inspection.

reference_spacy_tokens: The DHS report describes the three methods currently used by customs officials for the inspection of vehicles : a radiation portal monitor for the detection of radiation , a vehicle and cargo inspection system that creates an image similar to an X - ray , and a physical inspection .


fn_spacy_tokens_for_baseline: It is frequently seen in stormy weather off the Cape of Good Hope and considered an omen of ill fortune .

source_bert_toks: it is frequently seen in stormy weather off the cape of good hope and considered an om ##en of ill fortune .

reference_raw: It is often seen in stormy weather outside the Cape of Good Hope and is considered a sign of bad luck.

reference_spacy_tokens: It is often seen in stormy weather outside the Cape of Good Hope and is considered a sign of bad luck .


fn_spacy_tokens_for_baseline: The chairman , an unsympathetic Englishman , then mounted the rough platform which had been quickly tacked together that morning .

source_bert_toks: the chairman , an un ##sy ##mp ##ath ##etic englishman , then mounted the rough platform which had been quickly tack ##ed together that morning .

reference_raw: The Chairman, a dissimilar Englishman, climbed onto a rough platform that had been quickly taken together that morning.

reference_spacy_tokens: The Chairman , a dissimilar Englishman , climbed onto a rough platform that had been quickly taken together that morning .


fn_spacy_tokens_for_baseline: He patiently unravelled the bootlaces and Richard winced with pain as the circulation returned to his fingers .

source_bert_toks: he patiently un ##rave ##lled the boot ##lace ##s and richard winced with pain as the circulation returned to his fingers .

reference_raw: Patiently he unfolded the laces, and Richard winced with pain as the blood flowed back into his fingers.

reference_spacy_tokens: Patiently he unfolded the laces , and Richard winced with pain as the blood flowed back into his fingers .


fn_spacy_tokens_for_baseline: With the party buoyant in the opinion polls , and taking full benefit from government discomfiture over the poll tax , Labour increased its share of the vote to almost 55 % .

source_bert_toks: with the party bu ##oya ##nt in the opinion polls , and taking full benefit from government disco ##m ##fi ##ture over the poll tax , labour increased its share of the vote to almost 55 % .

reference_raw: With the party buoyant in the polls and taking full advantage of the government's displeasure over the polling tax, Labour increased its share of the vote to almost 55%.

reference_spacy_tokens: With the party buoyant in the polls and taking full advantage of the government 's displeasure over the polling tax , Labour increased its share of the vote to almost 55 % .


fn_spacy_tokens_for_baseline: This prevents the cork drying up , shrinking and letting in air thus oxidising the wine .

source_bert_toks: this prevents the cork drying up , shrinking and letting in air thus ox ##idi ##sing the wine .

reference_raw: This prevents the cork from drying, shrinking and letting in air, thereby oxidizing the wine.

reference_spacy_tokens: This prevents the cork from drying , shrinking and letting in air , thereby oxidizing the wine .


fn_spacy_tokens_for_baseline: Um , but I think if we quit , uh , building these Taj Mahals with the color TVs and sixty dollars sixty thousand a year to keep an inmate in there - on a - on a - on a life sentence , we should start hanging them and get it over with and let 's just screwing up the system uh .

source_bert_toks: um , but i think if we quit , uh , building these ta ##j mahal ##s with the color tv ##s and sixty dollars sixty thousand a year to keep an inmate in there - on a - on a - on a life sentence , we should start hanging them and get it over with and let ' s just screwing up the system uh .

reference_raw: Um, but I think if we stop, uh, building these Taj Mahals with color TVs and $60,000 a year to keep a prisoner on a-a-life sentence, we should start hanging them up and get it over with and screw up the system.

reference_spacy_tokens: Um , but I think if we stop , uh , building these Taj Mahals with color TVs and $ 60,000 a year to keep a prisoner on a - a - life sentence , we should start hanging them up and get it over with and screw up the system .


fn_spacy_tokens_for_baseline: Once there , he held on to power for a long time , easily in successive periods of government , with unusual difficulty in opposition .

source_bert_toks: once there , he held on to power for a long time , easily in successive periods of government , with unusual difficulty in opposition .

reference_raw: Once there, he held on to power for a long time, easily during consecutive periods of government, with unusual difficulty in resisting.

reference_spacy_tokens: Once there , he held on to power for a long time , easily during consecutive periods of government , with unusual difficulty in resisting .


fn_spacy_tokens_for_baseline: Love is the mystical marriage celebrated in the bridal chamber of the heart with its matchless splendour .

source_bert_toks: love is the mystical marriage celebrated in the br ##idal chamber of the heart with its match ##less sp ##len ##dou ##r .

reference_raw: Love is a mystical marriage celebrated in the wedding chamber of the heart with its unmatched beauty.

reference_spacy_tokens: Love is a mystical marriage celebrated in the wedding chamber of the heart with its unmatched beauty .


fn_spacy_tokens_for_baseline: AUSTRIA , Once expected to waltz smoothly into the European Union , is elbowing its partners , treading on toes and pogo - dancing in a most un - Viennese manner .

source_bert_toks: austria , once expected to waltz smoothly into the european union , is elbow ##ing its partners , tread ##ing on toes and po ##go - dancing in a most un - vie ##nne ##se manner .

reference_raw: AUSTRIA, once expected to waltz smoothly into the European Union, elbows its partners, treads on their toes and pogo-dances in the most unvienne way.

reference_spacy_tokens: AUSTRIA , once expected to waltz smoothly into the European Union , elbows its partners , treads on their toes and pogo - dances in the most unvienne way .


fn_spacy_tokens_for_baseline: Grin stood at the top of the stairway and let out a piteous yowl .

source_bert_toks: grin stood at the top of the stairway and let out a pit ##eous yo ##wl .

reference_raw: Grin stood at the top of the stairs, exhaling a pitiful howl.

reference_spacy_tokens: Grin stood at the top of the stairs , exhaling a pitiful howl .


fn_spacy_tokens_for_baseline: As it grows , the grass is scattered with the dusky blue and magenta petals of meadow cranesbill and , as the hay ripens , seeding heads of sorrel colour its sunset - gold with red .

source_bert_toks: as it grows , the grass is scattered with the dusk ##y blue and mage ##nta petals of meadow cranes ##bill and , as the hay ripe ##ns , seed ##ing heads of so ##rre ##l colour its sunset - gold with red .

reference_raw: As it grows, the grass is littered with dark blue and magenta petals of meadow cranesbill and, as the hay matures, sorrel seeds color the sunset gold and red.

reference_spacy_tokens: As it grows , the grass is littered with dark blue and magenta petals of meadow cranesbill and , as the hay matures , sorrel seeds color the sunset gold and red .


fn_spacy_tokens_for_baseline: From the beginning , Las Vegas was built to serve travelers .

source_bert_toks: from the beginning , las vegas was built to serve travelers .

reference_raw: From the start, Las Vegas was built to serve the travellers.

reference_spacy_tokens: From the start , Las Vegas was built to serve the travellers .


fn_spacy_tokens_for_baseline: The place was thronging with all sorts -- rough working men , sailors , neatly dressed tradespeople , a few dandies .

source_bert_toks: the place was th ##rong ##ing with all sorts - - rough working men , sailors , neatly dressed trades ##pe ##ople , a few dan ##dies .

reference_raw: It was crowded with all sorts of things - rough workers, sailors, well-dressed merchants, a few dandies.

reference_spacy_tokens: It was crowded with all sorts of things - rough workers , sailors , well - dressed merchants , a few dandies .


fn_spacy_tokens_for_baseline: It was an irresistible lure , like the quack of a decoy duck .

source_bert_toks: it was an irresistible lure , like the qu ##ack of a deco ##y duck .

reference_raw: It was an irresistible lure, like a bait duck's charlatan.

reference_spacy_tokens: It was an irresistible lure , like a bait duck 's charlatan .


fn_spacy_tokens_for_baseline: Thieves have used a mechanical digger to smash their way into a supermarket and steal a safe .

source_bert_toks: thieves have used a mechanical digger to smash their way into a supermarket and steal a safe .

reference_raw: The thieves used a mechanical digger to break into the supermarket and steal the vault.

reference_spacy_tokens: The thieves used a mechanical digger to break into the supermarket and steal the vault .


fn_spacy_tokens_for_baseline: So when voters began deserting the CDS in droves , the People 's Party stood to benefit most from the defections .

source_bert_toks: so when voters began desert ##ing the cds in drove ##s , the people ' s party stood to benefit most from the defect ##ions .

reference_raw: So when the voters began to leave the CDS in the driveways, the People's Party stood to benefit most of the defects.

reference_spacy_tokens: So when the voters began to leave the CDS in the driveways , the People 's Party stood to benefit most of the defects .


fn_spacy_tokens_for_baseline: Straws in the wind : A baby boom in Moscow may be signaling the renewal of Russian optimism , according to the New York Times . California 's Supreme Court ruled that victims of age discrimination in the workplace have the same rights as victims of race and sex discrimination .

source_bert_toks: straw ##s in the wind : a baby boom in moscow may be signaling the renewal of russian optimism , according to the new york times . california ' s supreme court ruled that victims of age discrimination in the workplace have the same rights as victims of race and sex discrimination .

reference_raw: According to the New York Times, a child boom in Moscow may signal the restoration of Russian optimism. The California Supreme Court found that victims of age discrimination in the workplace have the same rights as victims of racial and sexual discrimination.

reference_spacy_tokens: According to the New York Times , a child boom in Moscow may signal the restoration of Russian optimism . The California Supreme Court found that victims of age discrimination in the workplace have the same rights as victims of racial and sexual discrimination .


fn_spacy_tokens_for_baseline: After being welcomed to the summit of an amazingly red Red Pike by the rain , we were happy to descend to Scarth Gap .

source_bert_toks: after being welcomed to the summit of an amazingly red red pike by the rain , we were happy to descend to scar ##th gap .

reference_raw: When we were greeted by the rain at the top of the incredibly red Pike, we were happy to go down to the Scarth Gape.

reference_spacy_tokens: When we were greeted by the rain at the top of the incredibly red Pike , we were happy to go down to the Scarth Gape .


fn_spacy_tokens_for_baseline: ' Can we fashion some sort of stretcher ?

source_bert_toks: ' can we fashion some sort of stretch ##er ?

reference_raw: 'Can we make a stretcher?

reference_spacy_tokens: ' Can we make a stretcher ?


fn_spacy_tokens_for_baseline: This uses a radio link code - named Flaming Arrow .

source_bert_toks: this uses a radio link code - named flaming arrow .

reference_raw: Uses a radio link code named Flaming Arrow.

reference_spacy_tokens: Uses a radio link code named Flaming Arrow .


fn_spacy_tokens_for_baseline: Scientists have found that the Neanderthal Man loved flowers , for he was buried on a mound of hollyhocks and phlox and fragrant pine needles .

source_bert_toks: scientists have found that the ne ##ander ##thal man loved flowers , for he was buried on a mound of holly ##ho ##cks and ph ##lo ##x and fra ##grant pine needles .

reference_raw: Scientists found that the Neanderthal Man loved flowers because he was buried in a pile of hollyhocks and phlox and scented pine needles.

reference_spacy_tokens: Scientists found that the Neanderthal Man loved flowers because he was buried in a pile of hollyhocks and phlox and scented pine needles .


fn_spacy_tokens_for_baseline: There 's nothing they like better than having a good chat about the latest bit of gossip : who is doing what , where , and why .

source_bert_toks: there ' s nothing they like better than having a good chat about the latest bit of gossip : who is doing what , where , and why .

reference_raw: There's nothing they like better than talking about the last piece of gossip: who's doing what, where, and why.

reference_spacy_tokens: There 's nothing they like better than talking about the last piece of gossip : who 's doing what , where , and why .


fn_spacy_tokens_for_baseline: Drizzle with olive oil and top with flakes of fresh Parmesan for an elegant starter .

source_bert_toks: dr ##iz ##zle with olive oil and top with fl ##akes of fresh par ##mes ##an for an elegant starter .

reference_raw: Drip with olive oil and top with fresh Parmesan flakes for an elegant starter.

reference_spacy_tokens: Drip with olive oil and top with fresh Parmesan flakes for an elegant starter .


fn_spacy_tokens_for_baseline: Trams rattled , taxis blared their horns , and everywhere skyscrapers rose like jagged teeth to touch the azure sky .

source_bert_toks: trams rattled , taxis b ##lar ##ed their horns , and everywhere skyscraper ##s rose like jagged teeth to touch the azure sky .

reference_raw: Trams rattled, taxis blew horns, and everywhere the skyscrapers rose like jagged teeth to touch the azure sky.

reference_spacy_tokens: Trams rattled , taxis blew horns , and everywhere the skyscrapers rose like jagged teeth to touch the azure sky .


fn_spacy_tokens_for_baseline: Her limbs were graceful , her whole body proportioned like that of a young goddess ; as for her breasts , an infatuated young man who had seen her in a small part at Robert 's Shield Theatre wrote her a letter quoting the Song of Solomon at length : Asshe threatened to horsewhip him .

source_bert_toks: her limbs were graceful , her whole body proportion ##ed like that of a young goddess ; as for her breasts , an in ##fat ##uated young man who had seen her in a small part at robert ' s shield theatre wrote her a letter quoting the song of solomon at length : ass ##he threatened to horse ##w ##hip him .

reference_raw: Her limbs were graceful, and her whole body was the size of a young goddess; as far as her breasts were concerned, the mad young man who had seen her in a small part at Robert's Shield Theatre had written a letter to her, quoting the Song of Solomon at last: Asshe threatened to ride him.

reference_spacy_tokens: Her limbs were graceful , and her whole body was the size of a young goddess ; as far as her breasts were concerned , the mad young man who had seen her in a small part at Robert 's Shield Theatre had written a letter to her , quoting the Song of Solomon at last : Asshe threatened to ride him .


fn_spacy_tokens_for_baseline: You plot along at a steady crawl wondering if you 'll ever reach that far - off doorway , let go of the joystick as soon as you get there , then promptly take another pace !

source_bert_toks: you plot along at a steady crawl wondering if you ' ll ever reach that far - off doorway , let go of the joy ##stick as soon as you get there , then promptly take another pace !

reference_raw: You plot with a steady climb and wonder if you'll ever get to the far door, release the joystick as soon as you get there, and then take another step at once!

reference_spacy_tokens: You plot with a steady climb and wonder if you 'll ever get to the far door , release the joystick as soon as you get there , and then take another step at once !


fn_spacy_tokens_for_baseline: The series of nuclear tests in 1995 - 96 prior to China 's signature of the Comprehensive Test Ban Treaty ( CTBT ) may have resulted in a smaller and lighter warhead design for the new generation of intercontinental ballistic missiles ( ICBMs ) now under development .

source_bert_toks: the series of nuclear tests in 1995 - 96 prior to china ' s signature of the comprehensive test ban treaty ( ct ##bt ) may have resulted in a smaller and lighter war ##head design for the new generation of intercontinental ballistic missiles ( ic ##bm ##s ) now under development .

reference_raw: The series of nuclear tests in 1995 - 96 before China signed the Comprehensive Test Ban Treaty (CTBT) could lead to a smaller and lighter design for the new generation of intercontinental ballistic missiles (ICBMs) currently under development.

reference_spacy_tokens: The series of nuclear tests in 1995 - 96 before China signed the Comprehensive Test Ban Treaty ( CTBT ) could lead to a smaller and lighter design for the new generation of intercontinental ballistic missiles ( ICBMs ) currently under development .


fn_spacy_tokens_for_baseline: As February approached and the first snow of winter drizzled down on Clapham Common , Blanche began to lose hope of finding new evidence .

source_bert_toks: as february approached and the first snow of winter dr ##iz ##zle ##d down on clap ##ham common , blanche began to lose hope of finding new evidence .

reference_raw: As February approached, and the first snow of winter fell on Clapham Common, Blanche began to lose hope of finding new evidence.

reference_spacy_tokens: As February approached , and the first snow of winter fell on Clapham Common , Blanche began to lose hope of finding new evidence .


fn_spacy_tokens_for_baseline: ' My heart fell to my boots , ' Henrietta told another , ' for I dreaded that he was going to choose me and he did . '

source_bert_toks: ' my heart fell to my boots , ' henrietta told another , ' for i dread ##ed that he was going to choose me and he did . '

reference_raw: "My heart fell to my shoes," Henrietta said to the other, "because I was afraid he would pick me and he would."

reference_spacy_tokens: " My heart fell to my shoes , " Henrietta said to the other , " because I was afraid he would pick me and he would . "


fn_spacy_tokens_for_baseline: We could spend that money , you know , for starving children that are starving or twelve million other things would be more useful than that .

source_bert_toks: we could spend that money , you know , for starving children that are starving or twelve million other things would be more useful than that .

reference_raw: We could spend that money, you know, on starving children, who are hungry, or 12 billion other things would be more useful.

reference_spacy_tokens: We could spend that money , you know , on starving children , who are hungry , or 12 billion other things would be more useful .


fn_spacy_tokens_for_baseline: However , the dates may be off by an average of only 3 days .

source_bert_toks: however , the dates may be off by an average of only 3 days .

reference_raw: However, the data can only be cancelled by three days on average.

reference_spacy_tokens: However , the data can only be cancelled by three days on average .


fn_spacy_tokens_for_baseline: The ruins of Jerusalem were briefly liberated , but , in the end , Jewish resistance to Rome was defeated with great loss of life .

source_bert_toks: the ruins of jerusalem were briefly liberated , but , in the end , jewish resistance to rome was defeated with great loss of life .

reference_raw: For a moment, the ruins of Jerusalem were freed, but in the end, the Jewish resistance to Rome was beaten by a great loss of life.

reference_spacy_tokens: For a moment , the ruins of Jerusalem were freed , but in the end , the Jewish resistance to Rome was beaten by a great loss of life .


fn_spacy_tokens_for_baseline: Earlier this month , the world body said it would relocate a limited number of support staff , an announcement that coincided with the release of a U. N. report chronicling surging violence around the country .

source_bert_toks: earlier this month , the world body said it would relocate a limited number of support staff , an announcement that coincided with the release of a u . n . report chronic ##ling sur ##ging violence around the country .

reference_raw: At the beginning of this month, the world body announced that it would move a limited number of supporters, a statement consistent with the release of a UN report of chronicling surging violence across the country.

reference_spacy_tokens: At the beginning of this month , the world body announced that it would move a limited number of supporters , a statement consistent with the release of a UN report of chronicling surging violence across the country .


fn_spacy_tokens_for_baseline: Pietro Miletti appeared , his elegant shoes and trousers bespattered with mud .

source_bert_toks: pietro mile ##tti appeared , his elegant shoes and trousers be ##sp ##atter ##ed with mud .

reference_raw: Pietro Miletti appeared, his elegant boots and pants stained with mud.

reference_spacy_tokens: Pietro Miletti appeared , his elegant boots and pants stained with mud .


fn_spacy_tokens_for_baseline: One of mine was an impersonation of Winston Churchill saying Mary Had A Little Lamb .

source_bert_toks: one of mine was an imp ##erson ##ation of winston churchill saying mary had a little lamb .

reference_raw: One of mine was Winston Churchill's imitation that Mary had a little lamb.

reference_spacy_tokens: One of mine was Winston Churchill 's imitation that Mary had a little lamb .


fn_spacy_tokens_for_baseline: This normativist concern came to a head with the publication in 1929 by Lord Hewart , the Lord Chief Justice , of a book entitled The New Despotism .

source_bert_toks: this norma ##tiv ##ist concern came to a head with the publication in 1929 by lord he ##wart , the lord chief justice , of a book entitled the new des ##pot ##ism .

reference_raw: This normative concern came to the forefront of the publication of a book called "New Despotism" by Lord Hewart, Lord Chief Justice, in 1929.

reference_spacy_tokens: This normative concern came to the forefront of the publication of a book called " New Despotism " by Lord Hewart , Lord Chief Justice , in 1929 .


fn_spacy_tokens_for_baseline: Mock war 's so uncivil !

source_bert_toks: mock war ' s so un ##ci ##vil !

reference_raw: The Mock War is so incivil!

reference_spacy_tokens: The Mock War is so incivil !


fn_spacy_tokens_for_baseline: His alias was Ebrahim Ebrahimian , a name variously mis - spelled by the players , which Secord supposed to be Turkish .

source_bert_toks: his alias was e ##bra ##him e ##bra ##him ##ian , a name variously mis - spelled by the players , which sec ##ord supposed to be turkish .

reference_raw: His nickname was Ebrahim Ebrahimian, a name that the players misspelled, and that Secord was supposed to be Turkish.

reference_spacy_tokens: His nickname was Ebrahim Ebrahimian , a name that the players misspelled , and that Secord was supposed to be Turkish .


fn_spacy_tokens_for_baseline: just as cliche haunted Henry 's daily journey to the train , his socks from Marks and Spencers ' , his regular nightly bedtime , his fondness for a cup of tea at ten thirty in the evening , just as he seemed to be destined to be as remorselessly English as the plane trees in the street outside or the homecoming commuters clacking through the twilight towards the village , so his one existential act ( had n't someone called it that ? ) seemed destined for suburban predictability .

source_bert_toks: just as cl ##iche haunted henry ' s daily journey to the train , his socks from marks and spencer ##s ' , his regular nightly bed ##time , his fond ##ness for a cup of tea at ten thirty in the evening , just as he seemed to be destined to be as remorse ##lessly english as the plane trees in the street outside or the homecoming commuter ##s cl ##ack ##ing through the twilight towards the village , so his one existent ##ial act ( hadn ' t someone called it that ? ) seemed destined for suburban predict ##ability .

reference_raw: just as cliché haunted Henry's daily trip to the train, his socks from Marks and Spencer's, his regular night's sleep, his fondness for a cup of tea at ten-thirty in the evening, as it seemed to him that he was meant to be as ruthlessly English as the plane trees on the street outside, or the commuters at home clattering in the dusk toward the village, so his one existential act (none) that?) seemed to be meant for suburban predictability.

reference_spacy_tokens: just as cliche haunted Henry 's daily trip to the train , his socks from Marks and Spencer 's , his regular night 's sleep , his fondness for a cup of tea at ten - thirty in the evening , as it seemed to him that he was meant to be as ruthlessly English as the plane trees on the street outside , or the commuters at home clattering in the dusk toward the village , so his one existential act ( none ) that ? ) seemed to be meant for suburban predictability .


fn_spacy_tokens_for_baseline: The other firefighter was treated for injuries he received to his hand , which was believed to have been caused by glass or a cat scratching him .

source_bert_toks: the other fire ##fighter was treated for injuries he received to his hand , which was believed to have been caused by glass or a cat scratching him .

reference_raw: The second fireman was healed for injuries to his arm, which were believed to have been caused by glass or a cat scratching him.

reference_spacy_tokens: The second fireman was healed for injuries to his arm , which were believed to have been caused by glass or a cat scratching him .


fn_spacy_tokens_for_baseline: Beyond , in the New Territories , are a mixture of high - rise suburban towns , ancient sites and walled villages , country parks , and farms with ducks and fish ponds .

source_bert_toks: beyond , in the new territories , are a mixture of high - rise suburban towns , ancient sites and walled villages , country parks , and farms with ducks and fish ponds .

reference_raw: Beyond, in the new territories, there is a mixture of high-rise suburban cities, old places and walled villages, rural parks, and duck- and fish-pond holdings.

reference_spacy_tokens: Beyond , in the new territories , there is a mixture of high - rise suburban cities , old places and walled villages , rural parks , and duck- and fish - pond holdings .


fn_spacy_tokens_for_baseline: The objective of having kiosks is they serve as communication points between the guards and the visitors or those who are approaching the building and deal with them according to their requirements .

source_bert_toks: the objective of having ki ##os ##ks is they serve as communication points between the guards and the visitors or those who are approaching the building and deal with them according to their requirements .

reference_raw: The purpose of having a kiosk is that it serves as a communication point between guards and visitors or those who approach the building and deal with it in accordance with their demands.

reference_spacy_tokens: The purpose of having a kiosk is that it serves as a communication point between guards and visitors or those who approach the building and deal with it in accordance with their demands .


fn_spacy_tokens_for_baseline: Two main factories located in the cities of Kanggye and Sakchu are reportedly tasked with the final preparation , filling , and distribution of CW agents in artillery shells from 80 mm and larger , as well as 240 mm caliber artillery rockets .

source_bert_toks: two main factories located in the cities of kang ##gy ##e and sa ##k ##chu are reportedly tasked with the final preparation , filling , and distribution of cw agents in artillery shells from 80 mm and larger , as well as 240 mm caliber artillery rockets .

reference_raw: Two major factories located in Kanggye and Sakchu are said to be responsible for the final preparation, filling and distribution of CW agents in artillery cartridges of 80 mm or more, as well as 240 mm in artillery rockets.

reference_spacy_tokens: Two major factories located in Kanggye and Sakchu are said to be responsible for the final preparation , filling and distribution of CW agents in artillery cartridges of 80 mm or more , as well as 240 mm in artillery rockets .


fn_spacy_tokens_for_baseline: Exhalations from the Earth extinguish the sun 's fire gradually every day and cause the darkness of night .

source_bert_toks: ex ##hala ##tions from the earth ex ##ting ##uis ##h the sun ' s fire gradually every day and cause the darkness of night .

reference_raw: The breaths of the Earth extinguish the sun's fire every day, causing the darkness of the night.

reference_spacy_tokens: The breaths of the Earth extinguish the sun 's fire every day , causing the darkness of the night .


fn_spacy_tokens_for_baseline: Its restored lobby is Hong Kong 's most elegant gathering place ; afternoon tea here is a visitors ' ritual .

source_bert_toks: its restored lobby is hong kong ' s most elegant gathering place ; afternoon tea here is a visitors ' ritual .

reference_raw: Its restored lobby is the most elegant gathering place in Hong Kong; afternoon tea is a visitor's ceremony.

reference_spacy_tokens: Its restored lobby is the most elegant gathering place in Hong Kong ; afternoon tea is a visitor 's ceremony .


fn_spacy_tokens_for_baseline: At the end , the Roman general Titus laid siege to Jerusalem in a.d . 70 , finally attacking its starved and weakened defenders .

source_bert_toks: at the end , the roman general titus laid siege to jerusalem in a . d . 70 , finally attacking its starved and weakened defenders .

reference_raw: At the end of the day, the Roman general, Titus, besieged Jerusalem at the age of 70, and finally attacked his starving and weakened defenders.

reference_spacy_tokens: At the end of the day , the Roman general , Titus , besieged Jerusalem at the age of 70 , and finally attacked his starving and weakened defenders .


fn_spacy_tokens_for_baseline: The road bent slightly to the right .

source_bert_toks: the road bent slightly to the right .

reference_raw: The road curved slightly to the right.

reference_spacy_tokens: The road curved slightly to the right .


fn_spacy_tokens_for_baseline: Where this involves a mix of ISDN and multimedia ( see p 52 ) , it will bring easy access to a combination of sound , text , video , data and graphics , and revolutionise desktop communications .

source_bert_toks: where this involves a mix of is ##d ##n and multimedia ( see p 52 ) , it will bring easy access to a combination of sound , text , video , data and graphics , and revolution ##ise desktop communications .

reference_raw: When combined with ISDN and multimedia (see p 52), this will provide easy access to a mix of audio, text, video, data and graphics and revolutionize desktop communication.

reference_spacy_tokens: When combined with ISDN and multimedia ( see p 52 ) , this will provide easy access to a mix of audio , text , video , data and graphics and revolutionize desktop communication .


fn_spacy_tokens_for_baseline: And that when they caw so stridently , it 's always a sign of impending death . '

source_bert_toks: and that when they ca ##w so stride ##ntly , it ' s always a sign of impending death . '

reference_raw: And that when they're so strong, it's always a sign of impending death.'

reference_spacy_tokens: And that when they 're so strong , it 's always a sign of impending death . '


fn_spacy_tokens_for_baseline: The farms round about had had to employ German prisoners for labour , and on the whole they were quite well behaved .

source_bert_toks: the farms round about had had to employ german prisoners for labour , and on the whole they were quite well behaved .

reference_raw: Farms around them had to hire German prisoners to work, and all in all, they were well-behaved.

reference_spacy_tokens: Farms around them had to hire German prisoners to work , and all in all , they were well - behaved .


fn_spacy_tokens_for_baseline: You drink your coffee .

source_bert_toks: you drink your coffee .

reference_raw: Have a cup of coffee.

reference_spacy_tokens: Have a cup of coffee .


fn_spacy_tokens_for_baseline: I would never , ever have said I was anorexic , until a neighbour and now a close friend , Anita , came out and labelled me one .

source_bert_toks: i would never , ever have said i was an ##ore ##xi ##c , until a neighbour and now a close friend , anita , came out and labelled me one .

reference_raw: I would never, never say I was anorexic until a neighbor and now a close friend, Anita, came out and called me one.

reference_spacy_tokens: I would never , never say I was anorexic until a neighbor and now a close friend , Anita , came out and called me one .


fn_spacy_tokens_for_baseline: I have read of folk suffering such hallucinations when they have imbibed too freely .

source_bert_toks: i have read of folk suffering such hall ##uc ##inations when they have im ##bib ##ed too freely .

reference_raw: I've read about people who suffer from such hallucinations when they've been imbedded too freely.

reference_spacy_tokens: I 've read about people who suffer from such hallucinations when they 've been imbedded too freely .


fn_spacy_tokens_for_baseline: At one point , President Bush phoned Massoud Barzani , president of the Kurdish region , in hope of getting the Kurds to compromise on Kirkuk .

source_bert_toks: at one point , president bush phone ##d mass ##oud bar ##zan ##i , president of the kurdish region , in hope of getting the ku ##rds to compromise on kirk ##uk .

reference_raw: At one point, President Bush called Massoud Barzani, President of the Kurdish region, in the hope that the Kurds might be able to make a deal with Kirkuk.

reference_spacy_tokens: At one point , President Bush called Massoud Barzani , President of the Kurdish region , in the hope that the Kurds might be able to make a deal with Kirkuk .


fn_spacy_tokens_for_baseline: Test feeding was used less often , and the ability of junior house staff to palpate a mass was considerably less , during 1988 - 91 than during 1974 - 7 .

source_bert_toks: test feeding was used less often , and the ability of junior house staff to pal ##pate a mass was considerably less , during 1988 - 91 than during 1974 - 7 .

reference_raw: The use of test feed was less frequent and the ability of young housekeepers to pat the weight was significantly lower in the period 1988-91 than in the period 1974-7.

reference_spacy_tokens: The use of test feed was less frequent and the ability of young housekeepers to pat the weight was significantly lower in the period 1988 - 91 than in the period 1974 - 7 .