I’m going to share my honest opinions, David. I’m not trying to be impolite or combative, but this survey and subsequent promotion doesn’t sit well. I’m writing below what I will share with my students when we discuss this survey.

My issues with the study don’t run contrary to your expressed ‘best practices’ for blogger relations. Those are good. Rather, my concerns deal with the presentation of the study and the methodology. I believe I’ve covered most of my concerns above, but I’ll extend my remarks here.

Most of all, I feel it is best practice to reveal a complete methodology when releasing such a study. In fact, I would go so far as to say it is a required practice. Without that, I always doubt the veracity of any survey. That is not present on the site, at least that I can find.

I note that you’ve devoted an entire site to what serves, at best, as the beginning work leading up to a focus group. 102 respondents? What was your potential respondent pool? How were they contacted? How did they respond? Where’s the questionnaire? I could go on. Where’s the methodology?

If this is truly the < a target="_blank" href="http://dictionary.reference.com/search?q=heuristic">heuristic attempt at research and discussion, no one else may repeat your effort and add to the body of knowledge without the methodology. In education and PR, I believe that any effort to help others “learn, discover, understand, or solve problems on his or her own” requires this kind of disclosure.

I hear your request for participation in a conversation, but your site does not allow comments … except through the “Send us your questions & comments” link. That process only refers to the questions you used on past and present surveys.

The site’s implementation serves more as a barrier to conversation, IMO. You actually require people to sign up just to post a comment. Some people see that as a way to inhibit conversation … and even a tool for e-mail address harvesting, among other things. So, to see that in a site that professes free open access and discussion is a bit strange.

I realize you may be seeking to avoid spam, but there are other ways to avoid that problem.

Honestly, your request for help – but failure to share – looks more like trying to get participants to help you do your homework (for lack of a better term) and extend your own practice … not improving the broader practice and adding to the overall body of knowledge for others.

I’ll send my students there, to be sure. But, they likely won’t see the site as a place to interact with PR pros (which is what we tend to look for). Comments on the particular issues of survey preparation and analysis/presentation of results have no place to go. “The Findings” page has no opportunity for comment. Sub pages also have nothing … well, except for the very muted and hard to find tiny type of “Contact Us” in the footer of all pages.

You have plenty of promotional items like social bookmarks and e-mail to a friend, but little outreach to engage in a threaded conversation. Why did you choose that path?

We’ll visit, but right now … it really serves as a case study for what not to do in a survey and site implementation. As I said before, I’m truly not trying to be impolite. But I believe you can understand how, as an educator, I take these issues kinda seriously.

Finally, “scientific survey” is a misnomer. “Statistically significant” serves the purpose well. Your survey may be statistically significant for 102 people, should you survey them again (and again). We can’t repeat the process on any population (to see if it is significant for them) because we don’t have the methodology. Your survey results are not, and will never be, statistically significant for the broader PR professional population, nor the broader blogger population. We’re talking about populations in the US alone of tens of thousands for PR practitioners and millions for bloggers. 102 just won’t cut it.

I want to stress that we aren’t promoting this as a scientific survey. We’re trying to start a discusssion. We did actually take the time to reach out to bloggers who were ranked in the top 400 by Technorati and The Truth Laid Bear, but we’re getting the word out to broaden this discussion and include people like you and your readers.

The questions we developed were based on an earlier series of discussions and meetings between bloggers and PR professionals. But we’re not so arrogant to think that this group speaks for all bloggers or for all PR executives. In fact, we want to know if these are even the right questions to ask. You should know that the responses we got seemed to align with the attitudes and opinions I hear every day as a PR practitioner who specializes in social media. But the plural of “anecdote” is not “data,” as I’m sure you know.

We used the responses we received to start a discussion, not end it. Now comes the important part – we need people like you to visit the site and help us determine what questions we should ask on a broader scale. We also need help developing best practices for the profession. We want to use social media tools and principles to help us do this.

I’ve read your blog before and I respect what you have to say. I sincerely hope you and your students will be interested enough to participate in this process. Our hope is not to make this a scientific report, but an ongoing discussion that evolves as quickly as social media tools do.

Everyone is welcome here. Please help those of us who do this for a living develop and adhere to best practices.

Again, many thanks for your feedback. I really do want to hear from you.

Best,

David Wescott

Mississippi State’s Writing Project, the University of Arkansas Parents site, and The Ohio State University site would be cool to see. Maybe it is time for me to do some homework and find out who the contacts are there. Love to chat with them. It would make a great podcast, too.

At the very least they could have gone through some of the ranking directories like Technorati, for example, and sampled 100 from each category … like: Business, Entertainment, Lifestyle, Politics, Sports, and Technology. But, then that would have taken a bit of work, wouldn’t it.

And, they could have recognized that despite their 100+ council membership, there are actually thousands of PR firms and in-house PR operations in the US, alone. Sampling those would have helped, too. Still, more work.

What bothers me even more is the flimsy reporting of methodology and their propensity to (dare I say it) spin the results through cute word choices … like their use of percentages instead of the meager raw frequencies their survey actually produced. Oh, and their tendency to use universals & generalizations like “…sought to determine how bloggers and PR pros can improve…” also bothers me. They should have said “how 47 bloggers and 55 PR pros” to be more transparent and honest.

I can hear it now, “Hey, 59 percent sounds much better than actually revealing we’re talking about 27 people out of a potential respondent pool of millions.”

OK, who am I kidding. They probably wouldn’t speak like that. Respondent pools and frequencies may be new terms for them.

It makes me crazy.