Report
Evaluation Metrics for Measuring Bias in Search Engine Results
| العنوان: | Evaluation Metrics for Measuring Bias in Search Engine Results |
|---|---|
| المؤلفون: | Gezici, Gizem, Lipani, Aldo, Saygin, Yucel, Yilmaz, Emine |
| المصدر: | 24, 2021, 85-113 |
| سنة النشر: | 2022 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Computer Science - Information Retrieval |
| الوصف: | Search engines decide what we see for a given search query. Since many people are exposed to information through search engines, it is fair to expect that search engines are neutral. However, search engine results do not necessarily cover all the viewpoints of a search query topic, and they can be biased towards a specific view since search engine results are returned based on relevance, which is calculated using many features and sophisticated algorithms where search neutrality is not necessarily the focal point. Therefore, it is important to evaluate the search engine results with respect to bias. In this work we propose novel web search bias evaluation measures which take into account the rank and relevance. We also propose a framework to evaluate web search bias using the proposed measures and test our framework on two popular search engines based on 57 controversial query topics such as abortion, medical marijuana, and gay marriage. We measure the stance bias (in support or against), as well as the ideological bias (conservative or liberal). We observe that the stance does not necessarily correlate with the ideological leaning, e.g. a positive stance on abortion indicates a liberal leaning but a positive stance on Cuba embargo indicates a conservative leaning. Our experiments show that neither of the search engines suffers from stance bias. However, both search engines suffer from ideological bias, both favouring one ideological leaning to the other, which is more significant from the perspective of polarisation in our society. |
| نوع الوثيقة: | Working Paper |
| DOI: | 10.1007/s10791-020-09386-w |
| URL الوصول: | http://arxiv.org/abs/2210.10517 |
| رقم الانضمام: | edsarx.2210.10517 |
| قاعدة البيانات: | arXiv |
| ResultId |
1 |
|---|---|
| Header |
edsarx arXiv edsarx.2210.10517 1043 3 Report report 1043.46936035156 |
| PLink |
https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&scope=site&db=edsarx&AN=edsarx.2210.10517&custid=s6537998&authtype=sso |
| FullText |
Array
(
[Availability] => 0
)
Array ( [0] => Array ( [Url] => http://arxiv.org/abs/2210.10517 [Name] => EDS - Arxiv [Category] => fullText [Text] => View record in Arxiv [MouseOverText] => View record in Arxiv ) ) |
| Items |
Array
(
[Name] => Title
[Label] => Title
[Group] => Ti
[Data] => Evaluation Metrics for Measuring Bias in Search Engine Results
)
Array ( [Name] => Author [Label] => Authors [Group] => Au [Data] => <searchLink fieldCode="AR" term="%22Gezici%2C+Gizem%22">Gezici, Gizem</searchLink><br /><searchLink fieldCode="AR" term="%22Lipani%2C+Aldo%22">Lipani, Aldo</searchLink><br /><searchLink fieldCode="AR" term="%22Saygin%2C+Yucel%22">Saygin, Yucel</searchLink><br /><searchLink fieldCode="AR" term="%22Yilmaz%2C+Emine%22">Yilmaz, Emine</searchLink> ) Array ( [Name] => TitleSource [Label] => Source [Group] => Src [Data] => 24, 2021, 85-113 ) Array ( [Name] => DatePubCY [Label] => Publication Year [Group] => Date [Data] => 2022 ) Array ( [Name] => Subset [Label] => Collection [Group] => HoldingsInfo [Data] => Computer Science ) Array ( [Name] => Subject [Label] => Subject Terms [Group] => Su [Data] => <searchLink fieldCode="DE" term="%22Computer+Science+-+Information+Retrieval%22">Computer Science - Information Retrieval</searchLink> ) Array ( [Name] => Abstract [Label] => Description [Group] => Ab [Data] => Search engines decide what we see for a given search query. Since many people are exposed to information through search engines, it is fair to expect that search engines are neutral. However, search engine results do not necessarily cover all the viewpoints of a search query topic, and they can be biased towards a specific view since search engine results are returned based on relevance, which is calculated using many features and sophisticated algorithms where search neutrality is not necessarily the focal point. Therefore, it is important to evaluate the search engine results with respect to bias. In this work we propose novel web search bias evaluation measures which take into account the rank and relevance. We also propose a framework to evaluate web search bias using the proposed measures and test our framework on two popular search engines based on 57 controversial query topics such as abortion, medical marijuana, and gay marriage. We measure the stance bias (in support or against), as well as the ideological bias (conservative or liberal). We observe that the stance does not necessarily correlate with the ideological leaning, e.g. a positive stance on abortion indicates a liberal leaning but a positive stance on Cuba embargo indicates a conservative leaning. Our experiments show that neither of the search engines suffers from stance bias. However, both search engines suffer from ideological bias, both favouring one ideological leaning to the other, which is more significant from the perspective of polarisation in our society. ) Array ( [Name] => TypeDocument [Label] => Document Type [Group] => TypDoc [Data] => Working Paper ) Array ( [Name] => DOI [Label] => DOI [Group] => ID [Data] => 10.1007/s10791-020-09386-w ) Array ( [Name] => URL [Label] => Access URL [Group] => URL [Data] => <link linkTarget="URL" linkTerm="http://arxiv.org/abs/2210.10517" linkWindow="_blank">http://arxiv.org/abs/2210.10517</link> ) Array ( [Name] => AN [Label] => Accession Number [Group] => ID [Data] => edsarx.2210.10517 ) |
| RecordInfo |
Array
(
[BibEntity] => Array
(
[Identifiers] => Array
(
[0] => Array
(
[Type] => doi
[Value] => 10.1007/s10791-020-09386-w
)
)
[Subjects] => Array
(
[0] => Array
(
[SubjectFull] => Computer Science - Information Retrieval
[Type] => general
)
)
[Titles] => Array
(
[0] => Array
(
[TitleFull] => Evaluation Metrics for Measuring Bias in Search Engine Results
[Type] => main
)
)
)
[BibRelationships] => Array
(
[HasContributorRelationships] => Array
(
[0] => Array
(
[PersonEntity] => Array
(
[Name] => Array
(
[NameFull] => Gezici, Gizem
)
)
)
[1] => Array
(
[PersonEntity] => Array
(
[Name] => Array
(
[NameFull] => Lipani, Aldo
)
)
)
[2] => Array
(
[PersonEntity] => Array
(
[Name] => Array
(
[NameFull] => Saygin, Yucel
)
)
)
[3] => Array
(
[PersonEntity] => Array
(
[Name] => Array
(
[NameFull] => Yilmaz, Emine
)
)
)
)
[IsPartOfRelationships] => Array
(
[0] => Array
(
[BibEntity] => Array
(
[Dates] => Array
(
[0] => Array
(
[D] => 19
[M] => 10
[Type] => published
[Y] => 2022
)
)
)
)
)
)
)
|
| IllustrationInfo |