Helping the NHS with Novel ANOVAs: No Mean Feat!

Donate to Joseph Ohler, Jr.!

Even the eggheads at government think tanks occasionally need outside assistance. Large agencies in particular are most prone to needing external expertise due to how niche some of the agency functions get. For example, the National Health Service (NHS) — the U.K. equivalent of the Department of Health and Human Services (DHHS) in the U.S. — has a Foundation Trust tasked with conducting studies related to public health.

James Morgan, the Mental Health Act Manager at the Leeds and York Partnership NHS Foundation Trust, recently asked for help by posting in the “Psychology Students Network” LinkedIn group the following stats-related question — hence removed but with the URL recorded via hyperlink for posterity:

United Kingdom National Health Service Solicitation RE: Validating FAS Study

Nov. 25, 2013

James Morgan

Hi, all.

I have been asked to propose an experiment related to prenatal alcohol assumption and working memory. The experiment I’ve designed has three groups: a control group; a group containing children whose mother drank within the recommended limits whilst pregnant; and a group containing children diagnosed with FAS.

I planned on giving each participant four tests: two testing the phonological loop capabilities and two testing the visual-spatial sketchpad. From there, I want to firstly compare the scores from tests measuring the same component to determine whether there is a statistical link – I presume this is testing for validity?

Then I want to combine each participants’ scores into two – one score representing phonological capabilities and one visual-spatial – and then compare these results across groups to determine if there’s a statistical difference in results.

My hypothesis is that as alcohol consumption increases, there will be a significant reduction in the power of working memory components. Could anybody possible (sic) explain the statistical tests I would need to do to achieve this?

Thank you! :)

I sent the following message in response to James Morgan’s inquiry. It appeared I had cornered the market on answering his question — no one else had responded! Then, three things happened:

1) The postback stalled when I attempted to answer in the discussion thread;

2) I re-submitted my answer only for the error message, “This discussion is no longer available,” to appear; and

3) I re-sent my response via the private message feature.

Nov. 25, 2013

Joseph Ohler, Jr.

You’ll use a series of 6 one-way ANOVAs to test for a mean score difference in any direction, namely whether there exists a significantly non-null variance that exceeds the chance variance within each testing condition:

1) Phonological Test A

2) Phonological Test B

3) Visual-Spatial Test A

4) Visual-Spatial Test B

5) Composite Phonological Score

6) Composite Visual-Spatial Score

The ANOVAs on the composite scores serve as a reliability check for your other ANOVAs because the scores they measure are based on the scores measured by the earlier ANOVAs. Equally important is to include the Tukey HSD post-hoc option on each ANOVA to run pairwise comparisons.

Remember that an ANOVA tests for a main effect over all values of a category; the Tukey values indicate which individual comparisons are significant and which are trivial. If the ANOVA is a line of best fit, then the Tukey is the closeup of local peaks and valleys along that line.

The Tukey values show whether the differences between pairs of conditions are in the same direction (in which case the ANOVA difference is high due to additive main effects between categories) or are in opposite directions (in which case the ANOVA difference is low or non-existent due to subtractive main effects between categories).

This means that even if your ANOVA’s F ratio is insignificant, nonetheless consider the Tukey values because you may very well have a multimodal main effect. As you may know, the regression mean or line of best fit for a multimodal distribution may be nearly flat so always keep an open mind that your set of observations might be from a non-normal distribution.

All things equal, use a one-way ANOVA with the Tukey options selected. Include the “descriptives” field to add your confidence intervals for each score; these indicate the range of values where the actual mean score of each categorical population is 95% likely to actually be (if testing against a 5% significance threshold).

Although James did not respond verbally, he promptly removed the discussion thread almost exactly at the time of my first attempt to send the message: Interpret that as you will. Perhaps Morgan was embarrassed by having to ask in the first place and wanted to cover his tracks.

To that, I say: There are no “stupid” questions about statistics because most people feel out-of-place when having to address the technical aspects of how substantial a difference among groups is really significant or within a margin of error. Experts from many fields frequently use a subcontractor to achieve the proper statistical analysis that will withstand scrutiny by university professors, journalists, and government inspectors alike.

There’s really no reason for James or anyone to feel ashamed of asking the question he did, but if James Morgan chose another method for evaluating results, then he’s welcome to comment, providing his email address corresponds to his position in the National Health Service. (Email syntax may vary among government agencies, and I’m familiar enough to know who’s probably faking and who has a 99% chance of being the actual person.)

December 23, 2013 Update: Mr. Morgan has informed me via LinkedIn that the ANOVA with Tukey test was just the solution, and I thank him for the follow-up!

That’s what I love about the Internet — a blogger with a day job can interact with, even collaborate with, people of high authority from Western countries. (I’d advise staying clear of the less thoroughly monitored states, though — you don’t know what characters you’ll meet, and verifying ID is sketchier.)

Donate to Joseph Ohler, Jr.!

Hit Holiday Single!

The most thorough reviews of job vacancies in the universe!
The most thorough reviews of job vacancies in the universe!

National Health Service, Jim Morgan, Trust Manager, Phonological Study, Visual-Spatial Study, FAS Study, ANOVA Explanation, Statistical Analysis, Tukey Values

Ho-Ho-Ho-Hum: The Seasonal Hit Single!

Donate to Joseph Ohler, Jr.!

Just in time for gift-giving on the Feast of St. Nicholas, I’ve debuted my hit seasonal single “Ho-Ho-Ho-Hum!” This can save your reputation as a generous giver if you forgot about St. Nick’s Day, so download the tune and spread the mirth! I’ve generously included lyrics below so that you may sing along and resolve any “mondegreen” disputes:

“Ho-Ho-Ho-Hum” Lyrics

Verse 1
Don’t worry about last year’s mistakes /
Because festive joy this season makes /
Hey Santa Claus, covered in ash /
If you give a gift, then make it cash
Ho-ho-ho-hum, ho-hum doldrums /
Dum-dum-dum-dum, ho-hum doldrums!
Verse 2
The surest way to a happy mood /
Is to eat a lot of Christmas food /
Hey Santa Claus, how much you weigh? /
Can you still fit into your sleigh?
Back to Refrain
Ho-ho-ho’ing my way / Through the ho-hum doldrums /
Got a place to stay / During ho-hum doldrums
Verse 3
Do us all a favor this Yuletide /
Be more polite, and not as snide /
Hey Santa Claus, can you pay the rent? /
If you save the money that you spent?
Back to Refrain

Top 40 Holiday Hit!

Hit Holiday Single!

Top 40 Holiday Single!

Information About St. Nicholas Day

To avoid muddying the issue, I won’t bother delineating between St. Nicholas and Santa Claus. Some communities treat the two as one and the same — with typical gifts being chocolate Santa figurines — whereas others go to lengths at keeping the more sectarian St. Nicholas as distinct from the ecumenical Santa Claus.

Therefore, the gifts parents and relatives bestow to young ones on St. Nick’s Day tend to be more serious, practical, and with Christian iconography than are the goodies given on Christmas Eve or Christmas Day. The website “Why Christmas?” describes celebrations by country; those nations also observing St. Nicholas Day include a description of both celebrations.

If you poke through the links, you’ll notice Holland and Czech Republic celebrate mostly on December 5th but give the children gifts on the morning of December 6th, when the rest of Western Europe and its former colonies observes the feast. Orthodox-dominated countries celebrate on December 19th and will treat you tersely — as depicted below — if you try to persuade them otherwise:

Saint Nicholas Ticked Off

Uh-oh -- Saint Nicholas is angry!!!

I found a well-rounded explanation that discusses both the tradition before it developed a dozen or so permutations and the modern Dutch variants such as St. Nicholas riding a boat from Spain into the Netherlands with soot-covered chimney climbers in tow. Scandinavian countries prioritize their own saints above St. Nicholas during December; the Feast of St. Thorlakur on December 23rd is but one example.

Although African traditions relating to St. Nick are less publicized, West African legend has it that if you’re good, the village elder gives you a gingko tree seed — and if you’re bad, then world-famous Nigerian pro wrestler Power Uti arrives on his rocket chariot, body slams you through the roof of your family hut, and leaves you a service bill.

The usual date of celebration is December 6th, in alignment with colonial traditions. This is distinct from the December 25th feast at which localized Nativity plays are performed and children play with firecrackers in full view of a regional Santa Claus.

As for other regions? East Africa is Islamicized enough where the Feast of St. Nick and other Christian traditions aren’t widespread, despite Levantine folk narratives about how one of the Three Magi was from Ethiopia. Mainland China and North Korea don’t acknowledge Christian legends except in secular stores, but Hong Kong sells a lot of cheap St. Nicholas-related stuff year-round. Antarctica celebrations depend on the beliefs of the scientists stationed there at the time.

I’m sure that after reading all this some of you will have comments, so post below and share this article with your friends!

Donate to Joseph Ohler, Jr.!

Christmas tune, holiday song, parody of consumerism, ho-hum doldrums

Hit Holiday Single!