The quality and reliability of datasets play a crucial role in the success of any project. Datasets are not immune to bias and quality issues, and if not dealt with, these can have consequences for data analysis.

Data Bias

Bias in datasets refers to unfair or unrepresentative elements that can skew the results. It can stem from various sources, such as data collection methods, sample selection, or societal prejudices.

I conducted cursory background research to minimise data bias, including reading through the World Economic Forum’s Global Gender Pay Gap Report.

This would assist when performing exploratory data analysis (EDA) to identify patterns and discrepancies within the dataset due to data bias.

Quality Issues

The datasets chosen for my project were from several high-quality sources. This was essential for generating reliable and valid results. While biases primarily focus on the representativeness of data, quality issues encompass various aspects, such as completeness of data, accuracy, and consistency.

Therefore it was essential to perform data cleaning of the data sets. While evaluating the data sets, I looked to identify and rectify missing values, inconsistencies or errors. Where required, I removed problematic instances, ensuring the dataset is as accurate and reliable as possible.

Impact on Project

Data bias can significantly impact the evaluation of datasets as it can lead to skewed conclusions and inaccurate representation of results. Suppose certain groups or demographics are over- or under-represented in the data. In that case, the analysis and conclusions drawn from the dataset may not reflect the actual characteristics or trends of the target population. Biased data can lead to misleading insights and incorrect decision-making.

Data bias also raises ethical concerns regarding the responsible use of data. Using biased data without acknowledging or addressing the biases can be ethically problematic, as it may result in harm or discrimination against specific individuals or groups.

This might impact the gender pay gap analysis as data bias can reinforce existing stereotypes and perpetuate societal prejudices. If the dataset contains biased information that aligns with stereotypes, the study may unintentionally reinforce those biases, have negative social implications and hinder progress towards equality and fairness.

Quality issues in datasets can significantly impact the evaluation process as they can introduce errors, inconsistencies, or inaccuracies in the dataset, leading to unreliable results. Incorrect or incomplete data can distort statistical analyses and modelling, potentially leading to incorrect conclusions or predictions. Unreliable results can undermine the credibility of the project and its findings.

This can also undermine the validity of the dataset, reducing its ability to measure the variables of interest accurately. Through this, the robustness of the analysis is weakened. Robustness is vital for ensuring that the findings hold under different conditions and maintain their integrity.