C4R

Community for Rigor

Better Science Every Day

Activity Demo and Feedback Session

C4R Annual Conference 2024

Zac Parker,

Johns Hopkins University METER Team,

Smith College METER Team,

and

C4R CENTER Team

This work was supported by the National Institute of Neurological Disorders and Stroke of the National Institutes of Health under Award Numbers {METER AWARD} and UC2NS128361

We are doing an SfN workshop!

• Monday, October 7th 3pm-5pm Central

• Introduce and promote the Community for Rigor.

• Demonstrate activities as an entry point into materials.

• FOR STUDENTS Provide brief instruction in causal thinking and randomization.

• FOR EVERYBODY ELSE Convince them they could pick up our materials for their courses.

Speakers: Konrad, April, Gundula, Sara, Tari, Shai Covering: Why care about rigor, what is C4R, How are we teaching causal thinking, how are we teaching randomization. Coordinating call next week.

c4r.io/c4rlinks

Session Goals

• Introduce concepts in causal thinking and randomization.

• Stress test activities intended to teach skills in
  causal thinking and randomization.

• Develop an understanding of what improvements CENTER should prioritize in the next month of work.

Look For

• Is enough context provided to engage with the interactive components?

• Am I able to do what I intuitively want to do with the interactive components?

• Am I receiving actionable instructions throughout the session?

• Am I being given the chance to see what each interactive component can do?

Causality

Phenomena are Related

Consider tides.

Tides are different at different times of day.

Tides are different at different times of day
during different phases of the moon.

Tides are different at different times of day during different
phases of the moon at different coastal locations.

We Need to Map Relationships

When are tides ideal to swim?

We Need to Map Relationships

When are tides ideal to swim?

Are some seasons better for swimming than others?

We Need to Map Relationships

When are tides ideal to swim?

Are some seasons better for swimming than others?

Are coastal biomes, e.g. the rocky
intertidal, impacted by changing tidal cycles?

We Need to Map Relationships

When are tides ideal to swim?

Are some seasons better for swimming than others?

Are coastal biomes, e.g. the rocky
intertidal, impacted by changing tidal cycles?

Is the lunar tide effect constant enough to
predict its impact on rising sea levels?

We Need to Map Relationships

When are tides ideal to swim?

Are some seasons better for swimming than others?

Are coastal biomes, e.g. the rocky
intertidal, impacted by changing tidal cycles?

Is the lunar tide effect constant enough to
predict its impact on rising sea levels?

How can we map cause-effect mechanisms?

Vertices

• A vertex represents a variable or measurable entity.

• e.g. in When are tides ideal to swim?,
  both tides and (swimming conditions) are variables
  that could be represented by vertices.

Edges

• An edge is a connection
  between vertices.

• e.g. in When are tides ideal to swim?,
  the phrase are ideal is the connective relationship
  between tides and swimming conditions.

Vertices + Edges = DAGs

• A Directed Acyclic Graph is composed of
  vertices and edges to describe a causal relationship.

• DAGs help us to map phenomena
  and relevant variables

• Edges must have a single direction

• No cycles allowed

Use a DAG to Untangle a Correlation

Did Ice Cream Cause Polio?

Can you unravel the connection?

Start the activity

view DAGs from Group 1

Feedback

• In your own words, what was this activity trying to do?

• What did you want this activity to do that it did not do?

• What future applications do you see for the components of this activity?

Use a DAG to brainstorm causal factors

Can you find some causes?

Start the activity

view DAGs from Group 1

Feedback

• In your own words, what was this activity trying to do?

• What did you want this activity to do that it did not do?

• What would you need to change about this activity to recommend it to a colleague?

Randomization

A cool new result

Rigorous Raven is conducting a study to see if the delivery of a treatment virtually would impact the survival of their patients.

Patients were enrolled and allowed to choose whether they would receive the treatment in-person or virtually.

They seem to have found a strong treatment effect in their study!

The treatment includes support for medication adherence for the patients.

Population: Patients between the ages of 20-80 with the same terminal condition and medication
Intervention: Virtual treatment
Comparator: In-person treatment
Outcome: Survival

A cool new result

Rigorous Raven is conducting a study to see if the delivery of a treatment virtually would impact the survival of their patients.

Patients were enrolled and allowed to choose whether they would receive the treatment in-person or virtually.

They seem to have found a strong treatment effect in their study!

The treatment includes support for medication adherence for the patients.

Population: Patients between the ages of 20-80 with the same terminal condition and medication
Intervention: Virtual treatment
Comparator: In-person treatment
Outcome: Survival

But Wait

How do I know the effect is real?

What if there is a confounding variable that causes the two groups to be different in some way,

and THAT is the cause of a difference in survival?

Taking a closer look

Let’s take a closer look at the population variables to see if there is a relationship with treatment group and survival.

age	BMI
household income	sex
education	smoking
physical activity

You will be assigned a variable to explore.

What relationships can we discover between the variables in the study?

What would have been the correct approach?

1. This study has a lot of potential confounders:

age	BMI
household income	sex
education	smoking
physical activity

2. We could assign individuals to groups in a randomized way in case one of these confounders influences participants’ likelihood to select a treatment option.

If the assignment to in-person or virtual treatment was randomized…

If the assignment to in-person or virtual treatment was randomized…

Then the actual difference in survival between the two groups and would instead look like this:

Feedback

• In your own words, what was this activity trying to do?

• What did you want this activity to do that it did not do?

• How have your own students struggled with learning about randomization?

Next: We Can Design a Randomized Study

This is a Population with Heart Disease

We Want to Study the Population

Coins have limits

Block Randomization for Study Design

Block Randomization helps us with the design of our study,

• assign individuals to treatments within each study arm (i.e. “block”)

• while ensuring balanced numbers for treatment levels (e.g. drug A, drug B, placebo).

Try running and then modifying this code to generate random assignments in blocks.

Place our population into groups

We Block Randomized!

Feedback

• In your own words, what was this activity trying to do?

• What did you want this activity to do that it did not do?

• What would you need to change about this activity to recommend it to a colleague?

Today we covered

• Using DAGs to untangle causal relationships.

• Using DAGs to brainstorm possible causes of a phenomena.

• How randomization can improve confidence in results.

• How block randomization can improve consistency in group allocations.

Slide Formatting

About

This slideshow uses Revealjs through Quarto - see https://quarto.org/docs/presentations/revealjs/ for details.

Text Formatting (Dark Background)

Basic text formatting uses Markdown syntax, see https://quarto.org/docs/authoring/markdown-basics.html

This is text formatted as a block quote. Blah blah blah

Code is formatted as 2, 4, 6.

Text between one pair of asterisks is italicized;
text between two pairs of asterisks is bolded;
and three pairs does both.

• bullet
• example

Text Formatting Experiment Causality Confirmation Confounding Befuddlement

Basic text formatting uses Markdown syntax, see https://quarto.org/docs/authoring/markdown-basics.html

This is text formatted as a block quote. Blah blah blah

Code is formatted as 2, 4, 6.

Text between one pair of asterisks is italicized;
text between two pairs of asterisks is bolded;
and three pairs does both.

Bulleted Lists

This style uses a plain slide title formatting.

• here is an item
• here is another item
• here is an item with the same bullet style in the output even though the markdown uses a different symbol
  • here is an item in a sub-list

Numbered Lists

1. numbered lists

2. work without having to change the number

   • To force an identation level,
   • use 4 spaces instead of 2

Callout Box

Note

This text appears in a callout box.

For more information about styling, see https://quarto.org/docs/authoring/callouts.html

This text is a little bit bigger.

Features

Speaker Notes

This slide has speaker notes. Press s to see them.

Buffalo buffalo buffalo

Citations

The included refs.bib file contains citation data that we can refer to, such as the “Strong Inference” paper from Platt (1964). Cited works automatically appear at the end of the document. See https://quarto.org/docs/authoring/footnotes-and-citations.html

This sentence includes a cross-reference to Section 7

Activity Embed

This type of preview link puts the content in an iframe on TOP of the slides. This allows interactivity without needing to fiddle with sizing.

• a
• b

Start the stats widget

Layouts and Fun Stuff

Columns

LEFT (34%)

Insert
a
linebreak
with 2 spaces at the end of a line.

RIGHT (66%)

::::

Fragments

Fade in

Fade out

Highlight red

Fade in, then out

Slide up while fading in

Slide Transitions

Transitions can be added to entering and exiting a slide. More at https://quarto.org/docs/presentations/revealjs/advanced.html#slide-transitions

Animating content

Animating content

Wrap-Up

Takeaways

1. Here is a takeaway

References

Platt, J. 1964. “Strong Inference.” Science.

Acknowledgments

Unit Title

Conceptualization

TBD

Writing

Software

Funding