The concept of inverse document frequency (IDF)

The concept of inverse document frequency (IDF) is part of weighting information upon retrieval. The IDF method is common for search engines, usually as part of the tf-idf (term frequency–inverse document frequency) weighting scheme. The tf-idf is a numerical statistic that is intended to reflect how important a word is to a document in a collection or corpus. The method is often used as a weighting factor in information retrieval and text mining. How it works depend on the number of times a word appears on a document, offset by the word in the corpus to help it adjust for the fact that some words appear more frequently in general. IDF was the contribution of Karen Spärck Jones, a British computer scientist whose interest was in natural language processing and information retrieval. She first introduced it in a 1972 paper.