Mathematics/Statistics/Data Characteristics

From Dev Wiki
< Mathematics‎ | Statistics
Revision as of 23:53, 13 May 2020 by Brodriguez (talk | contribs) (Add notation note)
Jump to navigation Jump to search

The following details some basic characteristics of data in statistics. See also Core Measurements.

Note that all of the following assume an ordered set of items.


Min

The Min is the lowest single unique value.

For example, given a set of [1, 1, 2, 2, 3, 3, 4, 4, 5, 5], 1 is the min.


Max

The Max is the highest single unique value.

For example, given a set of [1, 1, 2, 2, 3, 3, 4, 4, 5, 5], 5 is the max.


Percentile

A Percentile is the value that marks 25%, 50% or 75% in your data.

In other words, find the median of your data. This marks the 50th percentile.
This splits your data into two halves. At which point you find the median for each half, giving you the 25th and 75th percentile.

For example, given a set of [1, 2, 3, 4, 5, 6], we have the following percentiles:
25th - 2
50th - 3.5
75th - 5


Quantiles

A Quartile is a segment of the data when split up into four chunks. Note that the set must be ordered for this.

There are four distinct quartiles with unique names:
Quartile 1 (Q1) - The lowest 25% of numbers in the set.
Quartile 2 (Q2) - The second lowest 25% of numbers in the set.
Quartile 3 (Q3) - The second highest 25% of numbers in the set.
Quartile 4 (Q4) - The highest 25% of numbers in the set.

In other words, sort your data and divide it into quarters. Each quarter is a quartile, with Q1 being the lowest values and Q4 being the highest.

For example, given a set of [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], we have the following quartiles:
Q1 - [1, 2, 3]
Q2 - [4, 5, 6]
Q3 - [7, 8, 9]
Q4 - [10, 11, 12]

While the above example splits neatly, not all sets will perfectly divide into fourths.
In these cases, a more scientific approach is to find the percentiles, and then use these as the boundaries for the quartiles.
In instances where an odd number of values are being split (so the percentile could arguably be in either quartile), one of two methods are common:

  1. Include the percentile value in both quartiles.
  2. Exclude the percentile value from both quartiles.

For example, consider this:

Given a set of [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], we have the following percentiles:
25th - 3
50th - 6
75th - 9
 
If including, we first split into [1, 2, 3, 4, 5, 6] and [6, 7, 8, 9, 10, 11].
Then we split again to get our quartiles.
Q1 - [1, 2, 3]
Q2 - [4, 5, 6]
Q3 - [6, 7, 8]
Q4 - [9, 10, 11]
 
If excluding, we first split into [1, 2, 3, 4, 5] and [7, 8, 9, 10, 11].
Then we split again to get our quartiles:
Q1 - [1, 2]
Q2 - [4, 5]
Q3 - [7, 8]
Q4 - [10, 11]


Quantiles

Similarly to quartiles, a quantile splits up data into evenly sized sections. The difference is that a quartile is a specific type of quantile, that always divides into fourths. Meanwhile, a quantile can be split into any number of sections.

For example, given a set of data, we can determine a quantile for every 10%. In this case, we would have the following quantiles:

10th quantile - Indicates that 10% of the data is below it and 90% of the data is above it.
20th quantile - Indicates that 20% of the data is below it and 80% of the data is above it.
30th quantile - Indicates that 30% of the data is below it and 70% of the data is above it.
40th quantile - Indicates that 40% of the data is below it and 60% of the data is above it.
50th quantile - Indicates that 50% of the data is below it and 50% of the data is above it.
60th quantile - Indicates that 60% of the data is below it and 40% of the data is above it.
70th quantile - Indicates that 70% of the data is below it and 30% of the data is above it.
80th quantile - Indicates that 80% of the data is below it and 20% of the data is above it.
90th quantile - Indicates that 90% of the data is below it and 10% of the data is above it.
Note: Quantiles sometimes have different names, depending on the text. For example, what we define here as the "50th quantile" may sometimes be referred to in other texts as "the 0.5 quantile" or "the 50% quantile". Yay inconsistent notations!


Outliers

An outlier is an item that does not fit in with the rest. It's usually extremely low or extremely high, compared to the other values in the set.

For example, given a set of [1, 2, 3, 4, 50], 50 is an outlier, as it's much higher than the rest of the values.