PhiloType's base quiz has 36 questions. Eight questions for each of the four axes, 32 in total, determine the type, and the remaining 4 determine the tendency toward interpretation or intervention. Answers use a 7-point scale, and the quiz takes about 8 minutes. This article explains how the count of 32 questions for the four axes was decided, based on three things: psychometric research, the question counts of similar services, and the results of actually taking other tests.
Items per axis and reliability
On the number of items in a scale, Hinkin (1998) suggests 4–6 items per construct for a final scale, and 8–12 items for the item pool during development. Three items are regarded as the minimum for identifying a measurement model.
Adding items raises Cronbach's α, which expresses a scale's internal consistency. With r as the average inter-item correlation, the α for each number of items given by the Spearman-Brown prophecy formula is shown below. Inter-item correlations in personality scales generally fall in the range .20–.30.
| Average inter-item correlation | 4 items | 6 items | 8 items | 10 items | 12 items |
|---|---|---|---|---|---|
| r = .20 | .50 | .60 | .67 | .71 | .75 |
| r = .25 | .57 | .67 | .73 | .77 | .80 |
| r = .30 | .63 | .72 | .77 | .81 | .84 |
Established short personality scales show the same pattern between items per factor and reliability.
| Scale | Total items | Items per factor | α (domains) |
|---|---|---|---|
| TIPI | 10 | 2 | .40–.73 |
| BFI-2-XS | 15 | 3 | mean .59–.63 |
| Mini-IPIP | 20 | 4 | .69–.81 |
| BFI-2-S | 30 | 6 | mean .77–.78 |
| BFI-2 | 60 | 12 | mean .86 |
According to Soto and John (2017), the BFI-2-S, with 6 items per factor, retained 93% of the full version's predictive power, and the BFI-2-XS, with 3 items per factor, retained 84%.
PhiloType's questions are written so that the options at both poles look equally attractive. This design tends to lower inter-item correlations, and when r is around .20, about two more items than usual are needed to reach the same α. The number of questions per axis was therefore set at 8, two more than 6.
Agreement of type results
When a type is determined by splitting a continuous score in two at the middle, people near the boundary tend to get a different result each time they retake the test. Cohen (1983) showed that splitting a continuous variable at the median reduces the variance that can be explained to about .65 times when one variable is dichotomized, and to about .41 times when both are.
Assuming a bivariate normal distribution and an axis reliability of ρ, the proportion of people who land on the same pole when classified again is ½ + arcsin(ρ)/π. If the four axes are independent, the proportion for which all four letters match is that value to the fourth power.
| Axis reliability ρ | Agreement on one axis | Agreement on all four letters |
|---|---|---|
| .60 | 70% | 25% |
| .70 | 75% | 31% |
| .80 | 80% | 40% |
| .90 | 86% | 54% |
| .95 | 90% | 65% |
The official MBTI (Form M) has 93 items, and the α of its four indices is around .90. Even so, the proportion of people whose four-letter type matches completely on retest is reported as 65% after four weeks, and as 72.9% in another analysis. These observed values are higher than the theoretical values in the table above, and one reason given is that few responses fall near the middle. Respondents near the boundary, on the other hand, still tend to get unstable results even with more items.
With PhiloType's 32 questions, if inter-item correlations fall in the range .20–.30, axis reliability is .67–.77, agreement on one axis is 73–78%, and agreement on all four letters is 29–37%. With this in mind, the result page marks axes that are a close call.
Question count and completion rate
As the number of questions and the time required grow, the share of people who answer to the end falls. In SurveyMonkey's analysis of about 25,000 surveys, the completion rate was 89% for 10-question surveys and 79% for 40-question surveys. In another analysis by the same company, drop-off rises most steeply up to 15 questions and more gradually after that. Published material from Qualtrics and others states that completion rates fall by 5–20 points when a survey takes longer than 7–8 minutes, and drop sharply at around 9 minutes on mobile.
Galesic and Bosnjak (2009) varied the time required announced at the start of a survey and compared start rates and completion rates.
| Announced time | Start rate | Completion rate |
|---|---|---|
| 10 minutes | 75.0% | 68.2% |
| 20 minutes | 69.9% | 56.8% |
| 30 minutes | 62.4% | 46.8% |
The same study also reported that answers to later questions were faster, shorter and more uniform.
Question counts of similar services
Personality and type test services fall roughly into three bands by number of questions.
| Band | Questions | Examples | Characteristics |
|---|---|---|---|
| Viral | 8–20 | therapytips 8, Inner Quests 10, Digram Shindan 20 | Free. Makes no accuracy claims and ends with sharing the result |
| Middle | 30–70 | IDRlabs Philosopher Personality Test 35, 16Personalities about 60, Truity Big Five 60, Keirsey 70, 8values 70 | A free entry point. Robust enough to show percentages, and connects to paid detailed reports |
| Full | 90 or more | MBTI Form M 93, VIA 96, mgram 105, Truity TypeFinder 130, RHETI 144, CliftonStrengths 177 | Mostly paid, or backed by business or research use |
Services that show a 16-type result for free cluster in the range of 8–12 questions per axis, or 32–48 questions in total when converted to four axes.
What we found by taking the tests
We took several tests ourselves and checked their question counts and screen layouts.
- The Japanese version of 16Personalities has 60 questions, six per screen, over 10 steps. Answers use a 7-point scale with labels only at the two ends. Questions 31–36 are a run of rewordings close in meaning to earlier questions.
- IDRlabs' Philosopher Personality Test has 35 questions, one per screen. From around question 20, abstract statements follow one after another, and it was possible to work backward from a question's content to the answer it was looking for.
- A Japanese test that determines 16 types with 12 two-choice questions can be finished in about a minute. It has only 3 questions per axis, however, the same level as the BFI-2-XS in the table above.
Why 32 questions
In summary, a count of 32 questions meets the following three points.
- With 8 questions per axis and inter-item correlations of .20–.30, α is .67–.77, on par with short scales that use 4–6 items per factor (Mini-IPIP, BFI-2-S)
- The 32 questions for the four axes take about 7 minutes, and even the 36 questions including the 4 on interpretation and intervention take about 8 minutes, staying at the edge of the range where completion rates are said to drop sharply
- It sits at the lower end of the standard range for similar services that show a 16-type result for free
There are also things 32 questions cannot measure. To tell apart people whose answers shift with the situation, or whose self-image differs from how they actually respond, the same axis has to be asked about repeatedly from different angles. A detailed test of more than 100 questions will therefore be prepared separately and offered with the paid personal guide and retakes. The current 32 questions are also a provisional set chosen from 40 candidates, and the 8 questions with the highest correlations on each axis will be reselected based on future response data.
The four questions on interpretation and intervention
Separately from the four axes, 4 questions determine the tendency toward interpretation or intervention, a difference in how people within the same type relate to the world. Interpretation is the tendency to put the weight on understanding the world and finding meaning in it, and intervention is the tendency to put the weight on acting on the world and changing it.
With 4 questions and inter-item correlations of .20–.30, α is .50–.63, lower than the four axes with 8 questions each. The result page therefore does not include this result in the four-letter type and shows it as a tendency. The 4 questions were chosen provisionally from 8 candidates and, like the 32 questions for the four axes, will be reselected based on future response data.
Sources
- Hinkin (1998): https://journals.sagepub.com/doi/10.1177/109442819800100106
- Soto & John (2017), BFI-2: https://www.colby.edu/wp-content/uploads/2013/08/Soto_John_2017b.pdf
- Gosling, Rentfrow & Swann (2003), TIPI: https://gosling.psy.utexas.edu/wp-content/uploads/2014/09/JRP-03-tipi.pdf
- Donnellan et al. (2006), Mini-IPIP: https://www.researchgate.net/publication/7014171
- Cohen (1983): https://journals.sagepub.com/doi/10.1177/014662168300700301
- MBTI Form M supplement: https://asia.themyersbriggs.com/wp-content/uploads/2016/09/MBTI_FormM_Supp.pdf
- Erford (2025): https://onlinelibrary.wiley.com/doi/10.1002/jcad.70006
- Galesic & Bosnjak (2009): https://academic.oup.com/poq/article-abstract/73/2/349/1939196
- SurveyMonkey (number of questions and completion rates): https://www.surveymonkey.com/curiosity/survey_questions_and_completion_rates/
- SurveyMonkey (time to complete a survey): https://www.surveymonkey.com/curiosity/survey_completion_times/
- Qualtrics: https://www.qualtrics.com/articles/strategy-research/4-tips-for-preventing-drop-offs-in-surveys/
- 16Personalities: https://www.16personalities.com/articles/reliability-and-validity
- IDRlabs Philosopher Personality Test: https://www.idrlabs.com/philosopher-personality/test.php
- Truity TypeFinder: https://www.truity.com/test/type-finder-personality-test-new
- Truity Big Five: https://www.truity.com/test/big-five-personality-test
- Keirsey Temperament Sorter: https://link.springer.com/rwe/10.1007/978-3-319-24612-3_44
- 8values: https://8values.github.io/
- VIA Survey: https://www.viacharacter.org/about/translations/via-adult-survey-translations
- CliftonStrengths: https://support.gallup.com/hc/en-us/articles/4415092592275
- Enneagram Institute RHETI: https://www.enneagraminstitute.com/rheti/
- mgram: https://mgram.me/ja/about
- Digram Shindan: https://digram-shindan.com/
- therapytips.org: https://therapytips.org/personality-tests/philosophical-orientation-test
- Inner Quests: https://www.innerquests.app/quiz