The SeaGL CFP Reviewing Criteria

Our team of external reviewers (one of whom just returned from seeing the 🚀 Hugo Award Ceremony 🚀 in Finland!) are making great progress on reading and scoring all 140 of the proposals we received for this year’s program. As we’re doing this we thought to ourselves, “Selves, wouldn’t it be cool if a conference actually shared its proposal selection criteria? That might help a lot of people craft their future proposals.”

So here it is! What follows is more or less exactly the guidance we sent our reviewers for how to rate proposals. Hopefully it’s helpful for you, our reader and potential speaker, and perhaps even for future conferences looking to direct their review process.

Audience profile

The audience runs the gamut from experienced FOSS technologists to the uninitiated/students, leaning quite heavily to the latter rather than the former.

Therefore talks about how to scale your unicorn are less applicable than more introductory topics.

Which isn’t to say there’s no space for unicorn scaling type talks, just that they will be in the minority for the program.

This is also a much less corporate conference/audience than, say, SCALE/OSCON/ATO/etc. The event is free to attend, so it attracts those who are interested in or passionate about technology more than those who are getting their company to pay for them to attend for training purposes. This also means that most of the audience will be very local to the Seattle region.

Reviewing/voting criteria

• Our CFP system can’t do blind reviews, but try to do as blind as possible (the abstract/content is more important than the presenter, on first pass at least)
• Only vote on talks which you feel qualified to judge
  • Leave stars blank else
  • Exception: something which sounds so cool or so crappy that you feel compelled to vote
• Similarly, consider the proposer’s qualifications to present the material
  • For instance, if someone proposed a talk on mental health & has not shown either that they’ve trained in the field or have done enough research to speak from a position of authority on the topic
    • Mental health is just an example here; substitute “containerisation” or “Linux init systems” or what have you if that helps internalise this criterion
  • Please vote down proposals where the presenter lacks adequate qualifications
• Keep the audience needs front and center
  • Don’t simply go with, “Yeah, I would like to see this” but rather, “Yeah, the audience would like to see this.”
• No advertisements or sales pitches allowed
  • Please vote down anything which looks like it’ll be a pitch
• GNU/Linux/Free Software not required
  • Yes, it’s the Seattle GNU/Linux conference
  • No, we’re not bigots against non-Free (as in speech) technologies
  • That said, proposals predominantly about proprietary technologies should be voted down
• Proposals including questionable language/messaging should be voted down
  • Language/platform/community shaming
  • Colloquialisms based in or statements implying or outright stating racism, sexism, anti-LGBTQ-ism
  • Etc.
• Non-software tech is welcome
  • For instance, last year we had a very well received talk about WW1 technologies
• Naturally, if you proposed something, please don’t vote on your own talks

Final selection process

Once everyone has reviewed all of the talks, we’ll be getting together on a call to discuss them. The topics of discussion are currently up in the air (because we’re still voting on talks right now) but will probably include things like, “We received 5 good container talks and can’t accept them all. Which one(s) do we prefer?” and “This talk got voted down but I’d like to make a case for it to be included in the program.”

After the call it’s up to the program chair, leaning on the feedback from the external reviewers and in coordination with the official SeaGL program committee staff, to select which talks will become a part of this year’s 50 hours of scheduled programming. The program needs to be a balance of topics, speakers, and difficulty levels, so it may be that some otherwise highly-rated talks can’t be included lest they upset that balance. As well, we’re hoping (if possible) to limit each speaker to a single talk. This will allow us to give more people a chance to present. Naturally there may be exceptions to this rule, particularly as life situations cause selected speakers to need to drop out of the program as the event approaches. (eh, it happens; we don’t take it personally or hold it against you)

We’re aiming to notify all proposers on August 28th. Here’s hoping we can hit that target!

Feedback

If, on or around August 28th, you receive notification that your proposals were not selected, there are a few things you can do…

1. Please don’t take it personally! We have a lot of proposals and a limited number of speaking slots. Declining your proposal only means it didn’t fit in with the program we chose. It does not necessarily mean it’s a bad talk idea or that you’ve failed in any way.
2. You can ask for feedback. Email VM Brasseur, our program chair, and let her know the proposal on which you’d like feedback. She can’t promise that there will be feedback to give, but if there is then she’ll share it as soon as she can.
3. Remember that all program selections are final. We cannot and will not change our decisions on whether to include a talk or not.

This post is copyright VM Brasseur and licensed CC BY-SA.