Study shows racial bias in evaluating legal writing

April 25, 2014
Back to TopCommentsE-mailPrintBookmark and Share

When handing out the same memo to various law partners to critique under the guise of a study on the writing competencies of young attorneys, researchers discovered law partners found more errors in the memos they believed were written by an African-American attorney.

Nextions wanted to explore the idea of confirmation bias in racial perceptions of writing skills. It received 53 responses from partners who either looked at the same memo written by "Thomas Meyer," an African-American third-year associate from NYU Law School; or the memo written by "Thomas Meyer," a Caucasian third-year associate from NYU Law School.

In a previous study, researchers found evidence that supervising lawyers perceived African-American lawyers to be subpar in their writing skills as compared to their Caucasian counterparts. The study results released this month by Nextions confirmed that.

African-American Thomas Meyer’s memo averaged a 3.2/5 rating; Caucasian Thomas Meyer’s memo averaged a 4.1/5 rating. Comments were also generally more positive for Caucasian Thomas Meyer.

The memo contained 22 different errors: seven minor spelling/grammar errors; six substantive technical writing errors; five errors in fact and four errors in the analysis of the facts. The memo was sent to 39 Caucasian partners and 21 racial/ethnic minorities; 23 were women and 37 were men.

“We undertook this study with the hypothesis that unconscious confirmation bias in a supervising lawyer’s assessment of legal writing would result in a more negative rating if that writing was submitted by an African American lawyer in comparison to the same submission by a Caucasian lawyer. In order to create a study where we could control for enough variables to truly see the impact of confirmation bias, we did not study the potential variances that can be caused due to the intersection of race/ethnicity, gender, generational differences and other such salient identities. Thus, our conclusion is limited to the impact of confirmation bias in the evaluation of African American men in comparison to Caucasian men. We do not know (although we plan to study the issue in the very near future!) how this impact will splinter or strengthen when gender and/or other identities are introduced,” the researchers wrote in their summary.

“The data findings affirmed our hypothesis, but they also illustrated that the confirmation bias on the part of the evaluators occurred in the data collection phase of their evaluation processes – the identification of the errors – and not the final analysis phase. When expecting to find fewer errors, we find fewer errors. When expecting to find more errors, we find more errors. That is unconscious confirmation bias. Our evaluators unconsciously found more of the errors in the ‘African American’ Thomas Meyer’s memo, but the final rating process was a conscious and unbiased analysis based on the number of errors found. When partners say that they are evaluating assignments without bias, they are probably right in believing that there is no bias in the assessment of the errors found; however, if there is bias in the finding of the errors, even a fair final analysis cannot, and will not, result in a fair result.”

Take a look at the study and let me know what you think.
 

ADVERTISEMENT
  • Great to see Dr Arin moving on
    The author of this report was previously all about getting separate but equal restrooms for womyn attorneys (but not female secretaries), so it is really good to see her moving on from that: Reeves acknowledged that there may be a desire for inclusion; however, there is still inadequate support for it. An example she provided was a woman lawyer who did not have a designated restroom, as there were only two: one for women secretaries and the other for male lawyers." http://www.lewisu.edu/news/Newsarticle.htm?PArticleID=5722#.U1vG1Vda80M
    • Thinking about it
      Who circuluates a memo with the race of the author on it? What to expect next month, sexual preferences listed?
    • Does the study confirm unconscious bias on the part of the author?
      It seems to me that the study is a pretty thin one where it is possible that it reveals more about the unconscious confirmation bias of the author than of the subjects. Perhaps she has the belief that law partners are biased against minorities and women and used the information to confirm her belief. Looking at the errors detected by the reviewers, the only place where there was a significant difference in errors detected was in the spelling errors (2.9 for the "Caucasian" vs. 5.8 for the "African-American") whereas the error differences in technical writing (4.1 vs. 4.9) and facts (3.2 vs. 3.9). That could easily be interpreted to reach the conclusion that spelling counts and the reviewers have a more negative view about the work of poor spellers. Of course, the point can be made that the partners found more spelling errors in the work of the "African-American" associate based on their biases; however, by not doing a more thorough job, the author detracts from her point.
      • Direct hit
        I was thinking just the same, Larry. Who crunches her stats? What was the confidence interval? Will this "study" be peer reviewed? Is she not a "consultant" seeking to peddle her "services" in light of this thin social research? Is this science or mere rhetoric and manipulation parading as social science? I have to say that the probability is that it would never pass muster in even an undergrad class on social science. Rather sad to see such shody work promoted here, but I guess since it arrived at the "right" (make that "left") conclusions it is good to go, breaking news, justification for indicting the entire social order. Coming next month, how inherent biases cause us to discriminate against left handed typists.

      Post a comment to this story

      COMMENTS POLICY
      We reserve the right to remove any post that we feel is obscene, profane, vulgar, racist, sexually explicit, abusive, or hateful.
       
      You are legally responsible for what you post and your anonymity is not guaranteed.
       
      Posts that insult, defame, threaten, harass or abuse other readers or people mentioned in Indiana Lawyer editorial content are also subject to removal. Please respect the privacy of individuals and refrain from posting personal information.
       
      No solicitations, spamming or advertisements are allowed. Readers may post links to other informational websites that are relevant to the topic at hand, but please do not link to objectionable material.
       
      We may remove messages that are unrelated to the topic, encourage illegal activity, use all capital letters or are unreadable.
       

      Messages that are flagged by readers as objectionable will be reviewed and may or may not be removed. Please do not flag a post simply because you disagree with it.

      Sponsored by
      ADVERTISEMENT
      1. I like the concept. Seems like a good idea and really inexpensive to manage.

      2. I don't agree that this is an extreme case. There are more of these people than you realize - people that are vindictive and/or with psychological issues have clogged the system with baseless suits that are costly to the defendant and to taxpayers. Restricting repeat offenders from further abusing the system is not akin to restricting their freedon, but to protecting their victims, and the court system, from allowing them unfettered access. From the Supreme Court opinion "he has burdened the opposing party and the courts of this state at every level with massive, confusing, disorganized, defective, repetitive, and often meritless filings."

      3. So, if you cry wolf one too many times courts may "restrict" your ability to pursue legal action? Also, why is document production equated with wealth? Anyone can "produce probably tens of thousands of pages of filings" if they have a public library card. I understand this is an extreme case, but our Supreme Court really got this one wrong.

      4. He called our nation a nation of cowards because we didn't want to talk about race. That was a cheap shot coming from the top cop. The man who decides who gets the federal government indicts. Wow. Not a gentleman if that is the measure. More importantly, this insult delivered as we all understand, to white people-- without him or anybody needing to explain that is precisely what he meant-- but this is an insult to timid white persons who fear the government and don't want to say anything about race for fear of being accused a racist. With all the legal heat that can come down on somebody if they say something which can be construed by a prosecutor like Mr Holder as racist, is it any wonder white people-- that's who he meant obviously-- is there any surprise that white people don't want to talk about race? And as lawyers we have even less freedom lest our remarks be considered violations of the rules. Mr Holder also demonstrated his bias by publically visiting with the family of the young man who was killed by a police offering in the line of duty, which was a very strong indicator of bias agains the offer who is under investigation, and was a failure to lead properly by letting his investigators do their job without him predetermining the proper outcome. He also has potentially biased the jury pool. All in all this worsens race relations by feeding into the perception shared by whites as well as blacks that justice will not be impartial. I will say this much, I do not blame Obama for all of HOlder's missteps. Obama has done a lot of things to stay above the fray and try and be a leader for all Americans. Maybe he should have reigned Holder in some but Obama's got his hands full with other problelms. Oh did I mention HOlder is a bank crony who will probably get a job in a silkstocking law firm working for millions of bucks a year defending bankers whom he didn't have the integrity or courage to hold to account for their acts of fraud on the United States, other financial institutions, and the people. His tenure will be regarded by history as a failure of leadership at one of the most important jobs in our nation. Finally and most importantly besides him insulting the public and letting off the big financial cheats, he has been at the forefront of over-prosecuting the secrecy laws to punish whistleblowers and chill free speech. What has Holder done to vindicate the rights of privacy of the American public against the illegal snooping of the NSA? He could have charged NSA personnel with violations of law for their warrantless wiretapping which has been done millions of times and instead he did not persecute a single soul. That is a defalcation of historical proportions and it signals to the public that the government DOJ under him was not willing to do a damn thing to protect the public against the rapid growth of the illegal surveillance state. Who else could have done this? Nobody. And for that omission Obama deserves the blame too. Here were are sliding into a police state and Eric Holder made it go all the faster.

      5. JOE CLAYPOOL candidate for Superior Court in Harrison County - Indiana This candidate is misleading voters to think he is a Judge by putting Elect Judge Joe Claypool on his campaign literature. paragraphs 2 and 9 below clearly indicate this injustice to voting public to gain employment. What can we do? Indiana Code - Section 35-43-5-3: Deception (a) A person who: (1) being an officer, manager, or other person participating in the direction of a credit institution, knowingly or intentionally receives or permits the receipt of a deposit or other investment, knowing that the institution is insolvent; (2) knowingly or intentionally makes a false or misleading written statement with intent to obtain property, employment, or an educational opportunity; (3) misapplies entrusted property, property of a governmental entity, or property of a credit institution in a manner that the person knows is unlawful or that the person knows involves substantial risk of loss or detriment to either the owner of the property or to a person for whose benefit the property was entrusted; (4) knowingly or intentionally, in the regular course of business, either: (A) uses or possesses for use a false weight or measure or other device for falsely determining or recording the quality or quantity of any commodity; or (B) sells, offers, or displays for sale or delivers less than the represented quality or quantity of any commodity; (5) with intent to defraud another person furnishing electricity, gas, water, telecommunication, or any other utility service, avoids a lawful charge for that service by scheme or device or by tampering with facilities or equipment of the person furnishing the service; (6) with intent to defraud, misrepresents the identity of the person or another person or the identity or quality of property; (7) with intent to defraud an owner of a coin machine, deposits a slug in that machine; (8) with intent to enable the person or another person to deposit a slug in a coin machine, makes, possesses, or disposes of a slug; (9) disseminates to the public an advertisement that the person knows is false, misleading, or deceptive, with intent to promote the purchase or sale of property or the acceptance of employment;

      ADVERTISEMENT