I found out I was shortlisted for Most Eclectic Taste shortly before getting on a plane and heading back to Germany, so I was unable to immediately react and express my excitement. After being nominated for a flurry of awards in the initial stage, I am of course thrilled that at least one stuck.So…do I fit the category? Does my blog? I like to think I have eclectic taste – I generally won’t turn down a great book just because it is in a genre I don’t usually read and I have been actively seeking out reads from less frequented genres as well.
These are the posts (all reviews) that I submitted for the category that reflect a range of genres from humor, speculative fiction, short story collections, non-fiction, children’s picture books, paranormal romance and literary fiction. I could have also chosen to highlight reviews of historical fiction, Christian fiction, health & beauty, or mystery and thrillers. It’s true that while my focus is YA and literary fiction, you never know what genre might pop up on my blog.
Humor
http://presentinglenore.blogspot.com/2008/12/book-review-santa-responds-by-santa.html
Short stories and non-fiction (marketing/business)
http://presentinglenore.blogspot.com/2009/01/book-reviews-for-mini-book-expo.html
Children's picture books and non-fiction (oceanography/science)
http://presentinglenore.blogspot.com/2009/05/hot-topic-deep-sea-monsters.html
YA paranormal
http://presentinglenore.blogspot.com/2009/05/book-review-swoon-by-nina-malkin.html
YA realistic fiction
http://presentinglenore.blogspot.com/2009/06/book-review-alive-and-well-in-prague.html
So am I the book blogger with the most eclectic taste? Now it is up to you decide – go to the BBAW awards voting booth now!
And now for some confessions of a BBAW panelist....
I’ve gotten a number of mails expressing outrage and disappointment that I didn’t make the best YA blog shortlist like I did last year and complaining about some of the ones that did. While I’m flattered that you all care so much, I don’t think anyone needs to “boycott BBAW” because of it.
First of all, BBAW is NOT just about the awards, which are just a tiny part of the whole celebration (check out the BBAW blog for all the events that will be going on and ways you can get involved).
Secondly, the 5 blogs that did shortlist in the YA category are all solid, deserving blogs that I read regularly and enjoy a great deal. Yes, my personal shortlist would be pretty different, but this a community process, and the community has spoken.
Thirdly, the fact I didn’t make this particular shortlist is my own fault. How? I simply ended up chosing the wrong links to submit – my chosen 5 were too long (most likely exasperating an already tired panelist), 4 out of the 5 were book reviews (perhaps perceived as not diverse enough), and none included pictures or video other than the book cover (probably earning a deduction).
I realized this as I was reading through the posts submitted in the panels I was judging and going through the line item scoring. You see, I submitted links not realizing the sheer amount of work the BBAW awards would turn out to be for the volunteers involved (way more than any of us on the awards committee ever bargained for – or we would have set it up differently, I assure you). In order to carefully read through every post submitted in most categories, BBAW awards panelists would have needed DAYS of reading blogs – time that most of us just did not have. Add the fact that some panelists were on more than one panel (I was on three, chairing one), and you get an idea of the immensity of the project.
And so, judges needed to skim at times and to develop shortcuts. Now, I don’t speak for any of the other panelists of course, but I can tell you a few of my own coping mechanisms.
For line items with a scoring value of 10 (such as quality of writing, originality, diversity of content, and fresh perspective) I assumed a start value of 7 (indicating an average blog). If a blog struck me as particularly good (or bad), I would add (or deduct points).
I also had a few automatic deductions. For example, if a blogger used the publisher summary in a book review instead of writing their own, I took off points for originality. If the summary went more than a paragraph, I deducted points off for not being concise. If the background of a blog was black or another dark (or too bright color), I could not say it was “easy on the eyes”.
I know there have been some complaints about nominated blogs not seeming to fit the category they were nominated in. Based on the line item scoring of 5 eligible posts, there was no way to disqualify blogs which submitted 5 posts that fulfilled the category requirements even if their focus lies elsewhere.
This is obviously not how every panelist approached judging, but I hope it shows what a time-consuming task it was and how we tried our best to be fair and consistent across judging. The system certainly wasn’t perfect, but I think we did end up with a fairly representative cross sample of the amazing book blogs that are out there. I bet by visiting some of the shortlisted blogs, you will find some new favorites. I know I did. And if you have any specific recommendations for how the awards process might be improved for next year, the awards committee welcomes your input. There will be a form at the end of BBAW that you can fill out, or if you need to get it off your chest now, feel free to e-mail me (lenoreva at hotmail dot com), Amy, or anyone on the 2009 awards committee.
PS: For more insight into the judging process, see this fab post over at the Book Smugglers.
Posts
37 comments:
Wow, I'm sure many didn't realize the amount of time and effort the judges put forth into picking the shortlisted blogs. Thanks to everyone who worked on BBAW!
You summed up the panelist experience very well. I was on three panels, and I read and judged more than 600 posts. I tried to give each post as fair a shake as I could.
I used a different but basically similar method of judging blogs. I assumed an average blog and added or subtracted points as necessary.
I have no idea why I didn't make more short lists or how close or far I was from making other lists, but I have no doubt that I was judged fairly.
I will work harder this year so I can be short-listed in a book review category, and I will likely be more careful about the specific posts I submit for judging.
I have two things to say to those who have complaints: (1) respectfully and in detail submit suggestions and solutions to the BBAW committee so next year will be better. Complaints without solutions aren't helpful. (2) Volunteer to serve on a committee, head a panel, or otherwise get involved. The more diverse the group of VOLUNTEERS (we were not appointees), the more BBAW will represent all book bloggers.
Oh and let me add a third: (3) Suggest more categories if you don't think your blog fit into an established category.
Sorry to have babbled on....
This post about BBAW is really interesting. It's nice to hear behind the scenes info, and I think it's good to know a bit more about how things work.
Thanks for this insightful post and as a panelist judging...it was time-consuming, but I wouldn't have volunteered if I didn't want to take the time out to do it.
I really enjoyed the process and it seems that the panel judging was fair all around...I assumed average scores for everyone and deducted based upon the actual posts submitted.
I served as a last-minute panelist, and as soon as I saw the criteria I realised that posts I'd picked were pretty much the opposite of them! So I'm totally with you on that. :)
I'm hoping that the critera get put up on the BBAW site, because then people might realise that they have great blogs that are just strong in other areas!
I've noticed places where people say that they knew their score in two genre categories and so chose to go with one rather than the other. I'm unclear how one could know one's scores at all. I didn't make the shortlist in a genre category to which I submitted posts and I'd be interested in seeing my scoring as help for knowing what an outside observer might be seeing or not seeing that could help me improve my blog.
Although I haven't commented on it or twittered about it, I also served on a panel. I worked on it everyday for hours and the day before scores were due judged in a 8 hour stretch. The scoring was so specific that even if a blog had great content could have been deducted points for lack of a book cover, difficult to subscribe, no way to contact, distracting sidebars, or impossible to read fonts. I also had no idea who the other judges were on my panel and we never discussed scoring together. I must admit that even I was surprised with who made the shortlists. My personal top fives of the panel I judged didn't all make the shortlist. In fact, the one I liked the most (which I had never even heard of until BBAW) didn't make it and one that wasn't even in my top 10 did. But the beauty of the process is that it's an average of all the judges. It's not just one person's opinions. Each judge had their own way of scoring but I trust that their scoring was consistent across the board.
Does the system need to be tweaked? Absolutely, I have opinions of my own. But like Beth mentioned, send feedback by email to the BBAW folks and if you have time, volunteer to help out. Everything in its infancy goes through its growing pains.
I think the BBAW is supposed to be fun right? I wasn't nomimated (and didn't expect to be) and I really don't care. (I wouldn't have complained though...) I think that the people that are complaining are taking themselves and the contest a little too seriously. I mean, most of us blog because we enjoy it. I don't know too many bloggers out there that are making a living out of this (yes, I know there are some, but I think that is such a small percentage it really doesn't count). I am grateful to BBAW because I've learned about so many new, exciting blogs that I never would have. I just can't believe that there are sour grapes after all the work that went into this. I can't even imagine how much reading was involved for those on the panels!
Thanks for the great post! I knew a lot of work went on behind the scenes, but it's nice to have some additional insight.
I was on two panels, and while they weren't among the longest, it was still A LOT of work. As I was telling someone in a comment recently, I think it's actually a good thing that nobody is 100% happy with the shortlists: it means that they don't correspond to any single individual's vision of what a good blog should be. The process wasn't flawless, but I know that Amy and all volunteers did the best they could. And next year will be better.
Pam - I don't think anybody knows what their scores were and were not given the option to shortlist in one genre category or another after the fact. I'd be curious to know where you saw people discussing what they knew their scores were.
In fact, I was nominated in 4 genre categories. If I wanted to ensure that I only shortlisted in the category that I wanted to shortlist in, I would have to withdraw my name for consideration from the three others BEFORE scoring started. Say, for example, I left all my nominations in, and happened to shortlist in four of them, I could not choose for myself which category I wanted to be in. The numbers decided.
Congratulations on making the shortlist! I was on two panels, so I know it was tons of work. I tired to spread the work out over a week's time so I wouldn't get burned out and all blogs would be scored fairly. I don't know who else was on my panels. I do think the system needs some tweaking, but I also think it's heads and shoulders above last year's system, so it's getting there.
I've avoided commenting about the "drama" around the BBAW judging process, but since I've seen post after post on the topic, I can't help but voice my opinion. And there's nothing in particular about your post, Lenore, that made me change my mind. It just happened to be the one I was reading at the time :)
I, too, was a judge for BBAW and judged two categories. It was an incredibly time-consuming process, one that I do not regret volunteering for. In introduced me to dozens of new blogs and bloggers.
After I finished the judging process, I took a few days "off" to recooperate. It was very disheartening after spending hours upon hours of judging blog posts to come back to the posts defaming BBAW and its organizers.
Many are posted by those that aren't informed of the process and are making obsurd claims that they can't back up. It is my suggestion that they simply ask for clarification about the process before posting false information about the panelists and the judging process.
Each panelist put a lot of time into reviewing and judging the blogs. We did not contact one another and attempt to influence one another's vote. We all put our personal feelings about the bloggers aside and judged based on the links that were submitted.
To go along with others that have commented, if you have problem, offer a suggestion. Don't blast BBAW, the panelists, the committee members, or Amy. Do a little research before posting information to make sure what you are posting is indeed accurate. And remember. There were over 1,000 individual blogs nominated. Each of these blogs had to be reviewed by someone. This is a very time-intensive process, and please take the feelings of those who performed these tasks into account before publicly taiting their names and reputations.
I knew it would be a long and potentially tedious process. You described it very well.
I hope people don't boycott BBAW because of foolishness like this. It's supposed to be about celebrating reading!
Found you on Twitter!
Congratulations on making the shortlist.
The judges did a phenomenal amount of work.
But I do wish that the criteria and scoring being used had been publicized to those of us who were put in the position of having to submit posts. Without that guidance, no doubt what was submitted may have even made the judges work that much more difficult.
Brimful - Thank you :)
Beth - Yeah, I read approx. 900 posts overall. It's likely the criteria will change for next year based on the learnings we've taken from this year's experiment. I'm not sure whether it is a good idea to say what the criteria are beforehand or not. That might make it too easy for people to work the system.
Janicu - I am all about transparency!
Serena - Yes! Despite the amount of work involved, it was definitely fun and rewarding being on the panels.
Eva - So true! Blogs are very individual and we all offer something great to the community.
Pam - No one knows how each individual panelist scored line items except that panelist. Chairs were sent final talleys so they could average scores to figure out the winners. I can tell you that on the panel I chaired, many blogs were very, very close to shortlisting. Maybe in the future the shortlists should include any blog that reaches a certain average? It's a thought.
My experience was similar. I was more of a deductor. It was also obvious to me when a blogger hadn't sent in their best links but only the last 5 posts. They'd be all memes or giveaways. I would think "I bet this poor soul has better stuff than this" but I was a stickler and only judged those posts. I'd say to anyone nominated next year: send in your own links or it's to your detriment.
Very interesting post Lenore!
I love to hear from "the other side of the BBAW" (I being a simple blogger). I think it's sad though, that people react so strongly to the shortlists and other matters relating to the event. Yes, a LOT of work comes into it (from reading your post and some answers, being a panelist is more like taking a second full-time job!), but in the end, it's supposed to be fun and celebrate our community!
Of course, my shorlists too would have been different in some parts, but that doesn't mean those who have been selected didn't deserve their place on there. :) That's also why I don't take it personally to not be on there. I know the competition is huge, both in quantity and quality. I think you are all doing a great job with the BBAW!
Wow! Well, first of all -- CONGRATS to you on being shortlisted!
Second, I didn't realize there was such an uproar about the shortlists. It really surprises me because the book blogging community (at least the YA one) is so accepting and supportive of each other. It seems very out of character. There are so many amazing blogs out there for every category, I can't imagine the task of narrowing it down to a list of 5 to vote on. THANKS to all the panelists who helped out and all the time you spent.
I'm sure that there are things that can be changed about the process, but I think the bones of it (open nominations, nominators submitting posts that represent the category for evaluation, and a blind panel scoring those posts) is a pretty good system. The YA panelists are probably not YA bloggers, so they don't have any pre-set opinions about which blogs they like or don't like. That helps to eliminate the "popularity" contest. It also gives good blogs that are not as well-known a chance to be recognized. Just because a blog has 800 followers and 1000 hits a day doesn't mean that they are necessarily the best blog. Conversely, a blog with only 100 followers may be a hidden jewel that people just don't know about yet.
If your blog didn't make the shortlist, maybe ask yourself "Why?" Maybe it just comes down to not selecting the "best" posts to represent your blog, like you said. Or maybe there are really things that can be improved?
I'm sorry that you didn't make the shortlist for best YA (because I love and read your blog regularly). But it doesn't mean that you don't have a top-notch blog. I'm also excited to have discovered some new YA blogs from the shortlist that I'd never seen before and are quickly becoming my new favorites!
Like you, I served on three panels and chaired one. My methods for judging the blogs were very similar to yours as well.
I was longlisted for four awards and didn't make the shortlists for any of them. I didn't expect to, and I wasn't surprised. Three of the four required me to submit links. I had a similar epiphany as I was applying the scoring criteria to the blogs I was judging - the posts I submitted weren't likely to get me the best scores. However, it was a learning experience - and a BIG nudge to do some re-design work on my blog :-).
Re: Chris' comment - Blogs that were judged on their last 5 posts may not have submitted links; the panel chair had to go to the blog to get them. Some of those blogs may have been at a disadvantage in the judging, but I saw some others that weren't hurt by it.
Great post, Lenore, and congrats on your nomination!
Congrats on making the shortlist. Well deserved.
I was on one panel, and I had to read and score more than 400 posts. I took the job very seriously and looked at each blog like I'd never seen them before. And many were new-to-me blogs. It was a very intensive process, and while I can see several areas where improvements need to be made for the future, overall I think it was fair and we did our best.
--Anna
Diary of an Eccentric
Florinda- There are definitely folks who blog A+ material all the time. But if some people's last 5 posts were memes they were at a disadvantage I think. I guess I'm saying that the ones complaining don't know what the judges were seeing. They might think so-and-so's blog rocks but it had to show in those 5 posts. It's also a case of you got to play to win so pick carefully, you know, if you want to.
congratulations on shortlisting "most eclectic," Lenore!
Your reflection on the panel/judging process is very similar to my experience. 2 panels, about 80 blogs each, with 5 posts each ... almost 800 posts!
I discovered a lot of new-to-me blogs that I've since added to my "must read" list (many of these didn't make the shortlist, but just getting nominated put them on my radar).
Your suggestion of making suggestions for improvements is a good one. I wonder if there's a debriefing committee to "strike while the iron is hot" (late Oct/Nov?) to round up volunteers to look at some of the suggestions for improvements that might be made in 2010.
Many thanks to you, all the bloggers, readers, panelists, chairs, and Amy for making BBAW possible.
Congratulations on making the shortlist! I will be glad to see your blog's name when I cast my vote.
I was on 2 panels and I judged the same way. I was actually surprised at who was shortlisted for both of mine, as they weren't the ones I'd most expected. Honestly, it's not surprising that people are unhappy with the shortlists. There are too many amazing blogs. Everyone seems to be complaining that their favorite blogs didn't make it, but even if they had all submitted fantastic posts, there wouldn't have been space. As it is, I feel the process is fair.
I also knew I wouldn't be shortlisted as soon as I saw the judging criteria. The posts I'd picked didn't match particularly well!
Congrats on the shortlist! It's an awesome feat no matter what category!
Congrats on being short listed! I really liked the way you explained your experiences with BBAW. I only served on one panel and that kept me very busy.
I was thinking that including the publisher's summaries on my reviews probably cost me some points. I like to use the publisher's summaries because they are available, easy to use, and I figure why should I re-create what's already been done?
I guess as long as this is going to be a requirement of BBAW awards criteria I probably won't have a good enough score to be short-listed. And I'm pretty much ok with that because I like the way I have my reviews set up.
Anyway, I am really looking forward to BBAW week. Best of luck in the Most Eclectic Taste category!
I thank you for giving us an inside look at how much work went into judging the blogs for the shortlist. I personally can't understand why people get all upset about this! This is just a way to express some appreciation for bloggers that really excel at certain things. I received several nominations but I knew that I would never get shortlisted. Just knowing that someone thought of me was an award for me!! At the end of the day if you are happy with your blog that is really all that matters. This is just icing on the cake!! :)
Congratulations on your nominations.
Thanks for the insider look at the short list process. It sounds a lot of work.
I'm really looking forward to BBAW so thanks for helping to make it possible.
Congrats! Great post. Thank you for showcasing your 5 posts, too. I very much enjoyed being a panelist for a category I'm not familiar with so I learned a lot; have a ton of new blogs to follow and many new books added to my tbr. I hope to take my voting even more seriously now that I know how much time and dedicated effort it took to tally. Here's to a wonderful BBAW. :)
Congrats on your nomination!
Interesting to know how the process worked. I honestly had no idea.
Now you have me thinking I need to change the look of my blog!! I just switched to a dark background!! I like my dark background!
One last thing...I appreciate how gracious most bloggers are being. Either humbled by their nominations or understanding in their defeat. I also like what Beth said, Don't complain if it isn't in a respectful manner and have a solution!
Thank you to all of the people who volunteered their time to the process!
Congrats on your nomination! Well deserved.
And thank you to all the volunteers who make BBAW a reality. :)
Moving forward, it seems like a good way to avoid as much "uproar" in the future might be to take the great insights on the judging process that you & others have put together, and create a nice little summary for the BBAW site. Maybe for each criteria, we could give 2-3 detailed bullets and/or examples.
That way, when folks are selecting their 5 links and they want to put the time in to make the right selections, they know where to aim. And then after the shortlists come out, there's something more concrete to point to & show that everything was applied fairly (not that it wasn't now, just that we don't have a common list to point to, y'know?).
Anyway, just my .02!
Kelly
Lenore,
Very interesting and very similar to the approach that I used in my review of some 200 blog posts for one category. Thank you for taking the time to show us your thought process.
Congrats on your shortlisting, and thanks for your work volunteering on the panel.
First of all - I don't think I congratulated you on being on the Most Eclectic blog short list...so congrats.
Secondly, thanks for this post explaining the panel role in the short lists. I was also a panel judge and it was a bit overwhelming. People need to understand this was the first year BBAW used panels like this...and I am sure there will be adjustments and changes from things we all learned from doing them.
Thirdly, anyone who would boycott this terrific event simply because their favorite blog didn't make a short list is missing the point of BBAW...it is about celebrating EVERYONE! The awards are only a very small part of the overall event. I congratulate everyone who helped Amy with putting on this HUGE event!
Post a Comment