dask.dataframe.DataFrame.describe

dask.dataframe.DataFrame.describe#

DataFrame.describe(split_every=False, percentiles=None, percentiles_method='default', include=None, exclude=None)[source]#

Generate descriptive statistics.

This docstring was copied from pandas.DataFrame.describe.

Some inconsistencies with the Dask version may exist.

Generate descriptive statistics.

Dask computes percentiles (used for the 25%, 50%, and 75% statistics) using an approximate algorithm by default. Results may therefore differ slightly from pandas. Use percentiles_method="dask" for the built-in Dask algorithm or percentiles_method="tdigest" for the t-digest algorithm. See dask.dataframe.DataFrame.quantile() for details.

Parameters:
split_everyint or False, optional

Number of partitions to aggregate at once. Defaults to False which uses a single-pass reduction over all partitions.

percentileslist-like of numbers, optional

The percentiles to include in the output. All should fall between 0 and 1. By default, [0.25, 0.5, 0.75] is used.

percentiles_method{“default”, “tdigest”, “dask”}, optional

Method for computing percentiles. "default" uses the internal Dask algorithm. "tdigest" uses the t-digest algorithm for floats and ints and falls back to "dask" otherwise.

Summarize the central tendency, dispersion, and shape of each
analyzed column’s distribution, excluding ``NaN`` values. By
default only numeric columns are analyzed; pass ``include`` to
also analyze non-numeric columns (or ``exclude`` to omit columns
by dtype).
Returns:
DataFrame

Summary statistics of the DataFrame’s columns.

See also

Series.describe

Generate descriptive statistics of a Series.

DataFrame.count

Count of non-NA observations per column.

DataFrame.max

Maximum of the values in each column.

DataFrame.min

Minimum of the values in each column.

DataFrame.mean

Mean of the values.

DataFrame.std

Standard deviation of the observations.

DataFrame.select_dtypes

Subset of a DataFrame including/excluding columns based on their dtype.

Notes

For numeric columns, the result’s index includes count, mean, std, min, max, and the requested percentiles. By default the lower percentile is 25 and the upper is 75; the 50 percentile is the same as the median.

For object columns, the result’s index includes count, unique, top, and freq. The top is the most common value and freq is its count. If multiple values tie for the highest count, top is chosen arbitrarily from among them.

With include='all', the result’s index is the union of the per-dtype indices, with NaN for statistics that do not apply to a given column’s dtype.

Examples

By default, only numeric columns are analyzed.

>>> df = pd.DataFrame(
...     {
...         "categorical": pd.Categorical(["d", "e", "f"]),
...         "numeric": [1, 2, 3],
...         "object": ["a", "b", "c"],
...     }
... )
>>> df.describe()
       numeric
count      3.0
mean       2.0
std        1.0
min        1.0
25%        1.5
50%        2.0
75%        2.5
max        3.0

All columns regardless of dtype.

>>> df.describe(include="all")
       categorical  numeric object
count            3      3.0      3
unique           3      NaN      3
top              d      NaN      a
freq             1      NaN      1
mean           NaN      2.0    NaN
std            NaN      1.0    NaN
min            NaN      1.0    NaN
25%            NaN      1.5    NaN
50%            NaN      2.0    NaN
75%            NaN      2.5    NaN
max            NaN      3.0    NaN

Restrict the result to a specific dtype.

>>> df.describe(include=["category"])
       categorical
count            3
unique           3
top              d
freq             1

Exclude a specific dtype.

>>> df.describe(exclude=[np.number])
       categorical object
count            3      3
unique           3      3
top              d      a
freq             1      1