Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -44,15 +44,15 @@ test: ### Runs all the project tests
$(PYTHON) -m pytest tests/

package: clean ### Runs the project setup
echo __version__ = \"$(version)\" > src/ydata_quality/__version__.py
echo __version__ = \"$(version)\" > src/data_quality/__version__.py
echo "$(version)" > VERSION
$(PYTHON) setup.py sdist bdist_wheel

install: ### Installs required dependencies
$(PIP) install dist/ydata-quality-$(version).tar.gz
$(PIP) install dist/data-quality-$(version).tar.gz

link-local: ### Installs the lib in dev mode
echo __version__ = \"$(version)\" > src/ydata_quality/__version__.py
echo __version__ = \"$(version)\" > src/data_quality/__version__.py
$(PIP) install -e .

upload:
Expand Down
53 changes: 25 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,30 @@
> **`ydata-quality` is now `data-quality`.** This package has been renamed to `data-quality`. Please follow the [Migration Guide](#migration-guide) as soon as possible — the old package will no longer receive updates or bug fixes.


[![](https://api.codacy.com/project/badge/Grade/f1662116e982402c956d2720fbd24507)](https://app.codacy.com/gh/ydataai/ydata-quality)
![](https://img.shields.io/github/workflow/status/ydataai/ydata-quality/release)
![](https://img.shields.io/pypi/status/ydata-quality)
[![](https://pepy.tech/badge/ydata-quality)](https://pypi.org/project/ydata-quality/)
[![](https://api.codacy.com/project/badge/Grade/f1662116e982402c956d2720fbd24507)](https://app.codacy.com/gh/Data-Centric-AI-Community/fg-data-quality)
![](https://img.shields.io/github/workflow/status/Data-Centric-AI-Community/fg-data-quality/release)
![](https://img.shields.io/pypi/status/fg-data-quality)
[![](https://pepy.tech/badge/fg-data-quality)](https://pypi.org/project/fg-data-quality/)
![](https://img.shields.io/badge/python-3.6%20%7C%203.7%20%7C%203.8-blue)
[![](https://img.shields.io/pypi/v/ydata-quality)](https://pypi.org/project/ydata-quality/)
![](https://img.shields.io/github/license/ydataai/ydata-quality)
[![](https://img.shields.io/pypi/v/fg-data-quality)](https://pypi.org/project/fg-data-quality/)
![](https://img.shields.io/github/license/Data-Centric-AI-Community/fg-data-quality)

# YData Quality
# Data Quality

ydata_quality is an open-source python library for assessing Data Quality throughout the multiple stages of a data pipeline development.
data_quality is an open-source python library for assessing Data Quality throughout the multiple stages of a data pipeline development.

A holistic view of the data can only be captured through a look at data from multiple dimensions and `ydata_quality` evaluates it in a modular way wrapped into a single Data Quality engine. This repository contains the core python source scripts and walkthrough tutorials.
A holistic view of the data can only be captured through a look at data from multiple dimensions and `data_quality` evaluates it in a modular way wrapped into a single Data Quality engine. This repository contains the core python source scripts and walkthrough tutorials.

## Quickstart

The source code is currently hosted on GitHub at: https://github.com/ydataai/ydata-quality
The source code is currently hosted on GitHub at: https://github.com/Data-Centric-AI-Community/fg-data-quality

Binary installers for the latest released version are available at the [Python Package Index (PyPI).](https://pypi.org/project/ydata-quality/)
Binary installers for the latest released version are available at the [Python Package Index (PyPI).](https://pypi.org/project/fg-data-quality/)
```
pip install ydata-quality
pip install fg-data-quality
```

### Comprehensive quality check in few lines of code

```python
from ydata_quality import DataQuality
from data_quality import DataQuality
import pandas as pd

#Load in the data
Expand Down Expand Up @@ -72,7 +69,7 @@ pip uninstall ydata-quality
### 2. Install the new package

```bash
pip install data-quality
pip install fg-data-quality
```

### 3. Update your imports
Expand All @@ -97,19 +94,19 @@ grep -r "ydata_quality" . --include="*.py"

## Examples

Here you can find walkthrough tutorials and examples to familiarize with different modules of `ydata_quality`
Here you can find walkthrough tutorials and examples to familiarize with different modules of `data_quality`

- [Start Here for Quick and Overall Walkthrough](https://github.com/ydataai/ydata-quality/blob/master/tutorials/main.ipynb)
- [Start Here for Quick and Overall Walkthrough](https://github.com/Data-Centric-AI-Community/fg-data-quality/blob/master/tutorials/main.ipynb)

To dive into any focussed module, and to understand how they work, here are tutorial notebooks:
1. [Bias and Fairness](https://github.com/ydataai/ydata-quality/blob/master/tutorials/bias_fairness.ipynb)
2. [Data Expectations](https://github.com/ydataai/ydata-quality/blob/master/tutorials/data_expectations.ipynb)
3. [Data Relations](https://github.com/ydataai/ydata-quality/blob/master/tutorials/data_relations.ipynb)
4. [Drift Analysis](https://github.com/ydataai/ydata-quality/blob/master/tutorials/drift.ipynb)
5. [Duplicates](https://github.com/ydataai/ydata-quality/blob/master/tutorials/duplicates.ipynb)
6. Labelling: [Categoricals](https://github.com/ydataai/ydata-quality/blob/master/tutorials/labelling_categorical.ipynb) and [Numericals](https://github.com/ydataai/ydata-quality/blob/master/tutorials/labelling_numerical.ipynb)
7. [Missings](https://github.com/ydataai/ydata-quality/blob/master/tutorials/missings.ipynb)
8. [Erroneous Data](https://github.com/ydataai/ydata-quality/blob/master/tutorials/erroneous_data.ipynb)
1. [Bias and Fairness](https://github.com/Data-Centric-AI-Community/fg-data-quality/blob/master/tutorials/bias_fairness.ipynb)
2. [Data Expectations](https://github.com/Data-Centric-AI-Community/fg-data-quality/blob/master/tutorials/data_expectations.ipynb)
3. [Data Relations](https://github.com/Data-Centric-AI-Community/fg-data-quality/blob/master/tutorials/data_relations.ipynb)
4. [Drift Analysis](https://github.com/Data-Centric-AI-Community/fg-data-quality/blob/master/tutorials/drift.ipynb)
5. [Duplicates](https://github.com/Data-Centric-AI-Community/fg-data-quality/blob/master/tutorials/duplicates.ipynb)
6. Labelling: [Categoricals](https://github.com/Data-Centric-AI-Community/fg-data-quality/blob/master/tutorials/labelling_categorical.ipynb) and [Numericals](https://github.com/Data-Centric-AI-Community/fg-data-quality/blob/master/tutorials/labelling_numerical.ipynb)
7. [Missings](https://github.com/Data-Centric-AI-Community/fg-data-quality/blob/master/tutorials/missings.ipynb)
8. [Erroneous Data](https://github.com/Data-Centric-AI-Community/fg-data-quality/blob/master/tutorials/erroneous_data.ipynb)

## Contributing
We are open to collaboration! If you want to start contributing you only need to:
Expand All @@ -123,7 +120,7 @@ You can also join the discussions on our [Discord Community](https://discord.com
For support in using this library, please join our Discord server. The Discord community is very friendly and great about quickly answering questions about the use and development of the library. [Click here to join our Discord community!](https://discord.com/invite/mw7xjJ7b7s)

## License
[GNU General Public License v3.0](https://github.com/ydataai/ydata-quality/blob/master/LICENSE)
[GNU General Public License v3.0](https://github.com/Data-Centric-AI-Community/fg-data-quality/blob/master/LICENSE)

## About

Expand Down
6 changes: 3 additions & 3 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
long_description = (here / 'README.md').read_text(encoding='utf-8')
version = (here / 'VERSION').read_text().rstrip("\n")

setup(name='ydata-quality',
setup(name='fg-data-quality',
version=version,
description='YData open-source tools for Data Quality.',
long_description=long_description,
Expand All @@ -34,8 +34,8 @@
'Topic :: Software Development :: Libraries :: Python Modules'
],
keywords='data science ydata',
url='https://github.com/ydataai/ydata-quality',
license="https://github.com/ydataai/ydata-quality/blob/master/LICENSE",
url='https://github.com/Data-Centric-AI-Community/fg-data-quality',
license="https://github.com/Data-Centric-AI-Community/fg-data-quality/blob/master/LICENSE",
python_requires=">=3.7, <3.9",
packages=find_namespace_packages('src'),
package_dir={'':'src'},
Expand Down
22 changes: 22 additions & 0 deletions src/data_quality/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
"""
YData open-source lib for Data Quality.
"""
from warnings import warn

from .core.data_quality import DataQuality

from .__version__ import __version__ # noqa: F401

warn(
"""
`import ydata_quality` is deprecated and will not receive more updates.
Please install data-quality via `pip install fg-data-quality` and use `import data_quality` instead.
""",
DeprecationWarning,
stacklevel=2,
)


__all__ = [
"DataQuality"
]
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
from pandas import DataFrame, Series
from dython.nominal import compute_associations

from src.ydata_quality.core.warnings import Priority
from src.data_quality.core.warnings import Priority

from ..core import QualityEngine, QualityWarning
from ..utils.correlations import filter_associations
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
from pandas import DataFrame
from numpy import argmin

from src.ydata_quality.core.warnings import Priority
from src.data_quality.core.warnings import Priority

from ..core import QualityEngine, QualityWarning
from ..utils.auxiliary import test_load_json_path
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

from numpy import argwhere, ones, tril
from pandas import DataFrame
from src.ydata_quality.core.warnings import Priority
from src.data_quality.core.warnings import Priority

from ..core import QualityEngine, QualityWarning
from ..utils.auxiliary import infer_dtypes, standard_normalize
Expand Down
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

from pandas import DataFrame

from src.ydata_quality.core.warnings import Priority
from src.data_quality.core.warnings import Priority

from ..core import QualityEngine, QualityWarning
from ..utils.auxiliary import find_duplicate_columns
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

from pandas import DataFrame

from src.ydata_quality.core.warnings import Priority
from src.data_quality.core.warnings import Priority

from ..core import QualityEngine, QualityWarning
from ..utils.enum import DataFrameType
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

from pandas import DataFrame, Series

from src.ydata_quality.core.warnings import Priority
from src.data_quality.core.warnings import Priority

from ..core import QualityEngine, QualityWarning
from ..utils.auxiliary import infer_dtypes
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

from pandas import DataFrame, Series

from src.ydata_quality.core.warnings import Priority
from src.data_quality.core.warnings import Priority

from ..core import QualityEngine, QualityWarning
from ..utils.correlations import filter_associations
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
2 changes: 1 addition & 1 deletion src/ydata_quality/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,4 @@

__all__ = [
"DataQuality"
]
]
2 changes: 1 addition & 1 deletion tests/engines/test_data_relations.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
import nbformat
from nbconvert.preprocessors import ExecutePreprocessor

from ydata_quality.data_relations.engine import DataRelationsDetector
from data_quality.data_relations.engine import DataRelationsDetector


@fixture(name='data_relations')
Expand Down
Loading
Loading