AgileDC 2023 Program Selection Process

Selecting session proposals for a conference program is a tough job. There are competing interests–speakers who want to present their ideas, companies who want to raise their visibility by having an employee speak, and attendees who want to learn and/or be entertained. 

The selection committee wants to satisfy the attendees, and also has an interest in crafting a program that presents information that is both useful and true. Having been a conference reviewer for many years and a number of conferences, I’ve seen my share of people new to a topic, extrapolating from insufficient experience, who propose sessions that sound like a terrible idea from my experience. There is enough misinformation and misleading guidance available on the Internet. If you need more, you can certainly ask ChatGPT.

When filtering out the chaff, it’s too easy to also filter out good information that just happens to be novel or unknown to the reviewers. When selecting known-good speakers offering known-good topics, it’s easy to fall into a rut. I assure you that most reviewers that I’ve known work hard to avoid these subjective biases, to the extent that a human can do so.

For AgileDC 2022 we used the “Powerball” random selection method to choose between submitted sessions. Our process was based on the work of Adam Cronkite of Democracy Research, and Mike Lauer of the National Institutes of Health (NIH), which is explored in the Revisionist History Podcast: The Powerball Revolution. The rationale is that random selection avoids subjective bias.

At AgileDC, as at NIH, we tempered that unbiased randomness with a filter to loosely judge a proposal’s “sufficiency and appropriateness” for a session at the conference. We try to avoid blatant sales pitches.

This worked well, but I felt that I might be shirking my responsibility as an experienced Agilist and reviewer by relying so heavily on the random selection. This year, we plan to inject a little more subjectivity in the form of “expert’s favorites.” The reviewers on the program committee, while judging sufficiency and appropriateness, will keep an eye out for proposals they think are especially good. We will discuss those at selection time, and perhaps select up to 8 sessions by consensus before continuing with the random selection.

The random selection process will proceed the same as last year. Each proposed speaker will be assigned a random number and then the list will be sorted by that number. The committee will work through that list in that sorted order, choosing one of each speaker’s submissions based on submission preference and on the desire to present a varied and balanced program. Where the committee does not have a clear favorite, they may consult with the speaker to help choose.

If a selected speaker must drop out for some reason, the committee will attempt to fill the slot with the next speaker in the random list.