(approximately 235,000) of the books were filtered out in this way. Table $1 lists the fraction removed at this stage for our other non-English corpora. 11.1D. Year Restriction In order to further ensure publication date accuracy and consistency of dates across all our corpora, we implemented a publication year restriction and only retained books with publication years starting from 1550 and ending in 2008. We found that a significant fraction of mis-dated books have a publication year of 0 or dates prior to the invention of printing. The number of books filtered due to this year range restriction is considerably small, usually under 2% of the original number of books. The fraction of the corpus removed by all stages of the filtering is summarized in Table $1. Note that because the filters are applied in a fixed order, the statistics presented below are influenced by the sequence in which the filters were applied. For example, books that trigger both the OCR quality filter and by the language correction filter are excluded by the OCR quality filter, which is performed first. Of course, the actual subset of books filtered is the same regardless of the order in which the filters are applied. I].2. Metadata based subdivision of the Google Books Collection II].2A. Determination of language To create accurate corpora in particular languages that minimize cross-language contamination, it is important to be able to accurately associate books with the language in which they were written. To determine the language in which a text is written, we rely on metadata derived from our 100 bibliographic sources, as well as statistical language determination using the Popat algorithm (Ref $3). The algorithm takes advantage of the fact that certain character sequences, such as ‘the’, 'of, and ‘ion", occur more frequently in English. In contrast, the sequences '‘la', 'aux', and 'de’ occur more frequently in French. These patterns can be used to distinguish between books written in Engl