Study shows racial bias in evaluating legal writing

April 25, 2014
Back to TopCommentsE-mailPrintBookmark and Share

When handing out the same memo to various law partners to critique under the guise of a study on the writing competencies of young attorneys, researchers discovered law partners found more errors in the memos they believed were written by an African-American attorney.

Nextions wanted to explore the idea of confirmation bias in racial perceptions of writing skills. It received 53 responses from partners who either looked at the same memo written by "Thomas Meyer," an African-American third-year associate from NYU Law School; or the memo written by "Thomas Meyer," a Caucasian third-year associate from NYU Law School.

In a previous study, researchers found evidence that supervising lawyers perceived African-American lawyers to be subpar in their writing skills as compared to their Caucasian counterparts. The study results released this month by Nextions confirmed that.

African-American Thomas Meyer’s memo averaged a 3.2/5 rating; Caucasian Thomas Meyer’s memo averaged a 4.1/5 rating. Comments were also generally more positive for Caucasian Thomas Meyer.

The memo contained 22 different errors: seven minor spelling/grammar errors; six substantive technical writing errors; five errors in fact and four errors in the analysis of the facts. The memo was sent to 39 Caucasian partners and 21 racial/ethnic minorities; 23 were women and 37 were men.

“We undertook this study with the hypothesis that unconscious confirmation bias in a supervising lawyer’s assessment of legal writing would result in a more negative rating if that writing was submitted by an African American lawyer in comparison to the same submission by a Caucasian lawyer. In order to create a study where we could control for enough variables to truly see the impact of confirmation bias, we did not study the potential variances that can be caused due to the intersection of race/ethnicity, gender, generational differences and other such salient identities. Thus, our conclusion is limited to the impact of confirmation bias in the evaluation of African American men in comparison to Caucasian men. We do not know (although we plan to study the issue in the very near future!) how this impact will splinter or strengthen when gender and/or other identities are introduced,” the researchers wrote in their summary.

“The data findings affirmed our hypothesis, but they also illustrated that the confirmation bias on the part of the evaluators occurred in the data collection phase of their evaluation processes – the identification of the errors – and not the final analysis phase. When expecting to find fewer errors, we find fewer errors. When expecting to find more errors, we find more errors. That is unconscious confirmation bias. Our evaluators unconsciously found more of the errors in the ‘African American’ Thomas Meyer’s memo, but the final rating process was a conscious and unbiased analysis based on the number of errors found. When partners say that they are evaluating assignments without bias, they are probably right in believing that there is no bias in the assessment of the errors found; however, if there is bias in the finding of the errors, even a fair final analysis cannot, and will not, result in a fair result.”

Take a look at the study and let me know what you think.
 

ADVERTISEMENT
  • Great to see Dr Arin moving on
    The author of this report was previously all about getting separate but equal restrooms for womyn attorneys (but not female secretaries), so it is really good to see her moving on from that: Reeves acknowledged that there may be a desire for inclusion; however, there is still inadequate support for it. An example she provided was a woman lawyer who did not have a designated restroom, as there were only two: one for women secretaries and the other for male lawyers." http://www.lewisu.edu/news/Newsarticle.htm?PArticleID=5722#.U1vG1Vda80M
    • Thinking about it
      Who circuluates a memo with the race of the author on it? What to expect next month, sexual preferences listed?
    • Does the study confirm unconscious bias on the part of the author?
      It seems to me that the study is a pretty thin one where it is possible that it reveals more about the unconscious confirmation bias of the author than of the subjects. Perhaps she has the belief that law partners are biased against minorities and women and used the information to confirm her belief. Looking at the errors detected by the reviewers, the only place where there was a significant difference in errors detected was in the spelling errors (2.9 for the "Caucasian" vs. 5.8 for the "African-American") whereas the error differences in technical writing (4.1 vs. 4.9) and facts (3.2 vs. 3.9). That could easily be interpreted to reach the conclusion that spelling counts and the reviewers have a more negative view about the work of poor spellers. Of course, the point can be made that the partners found more spelling errors in the work of the "African-American" associate based on their biases; however, by not doing a more thorough job, the author detracts from her point.
      • Direct hit
        I was thinking just the same, Larry. Who crunches her stats? What was the confidence interval? Will this "study" be peer reviewed? Is she not a "consultant" seeking to peddle her "services" in light of this thin social research? Is this science or mere rhetoric and manipulation parading as social science? I have to say that the probability is that it would never pass muster in even an undergrad class on social science. Rather sad to see such shody work promoted here, but I guess since it arrived at the "right" (make that "left") conclusions it is good to go, breaking news, justification for indicting the entire social order. Coming next month, how inherent biases cause us to discriminate against left handed typists.

      Post a comment to this story

      COMMENTS POLICY
      We reserve the right to remove any post that we feel is obscene, profane, vulgar, racist, sexually explicit, abusive, or hateful.
       
      You are legally responsible for what you post and your anonymity is not guaranteed.
       
      Posts that insult, defame, threaten, harass or abuse other readers or people mentioned in Indiana Lawyer editorial content are also subject to removal. Please respect the privacy of individuals and refrain from posting personal information.
       
      No solicitations, spamming or advertisements are allowed. Readers may post links to other informational websites that are relevant to the topic at hand, but please do not link to objectionable material.
       
      We may remove messages that are unrelated to the topic, encourage illegal activity, use all capital letters or are unreadable.
       

      Messages that are flagged by readers as objectionable will be reviewed and may or may not be removed. Please do not flag a post simply because you disagree with it.

      Sponsored by
      ADVERTISEMENT
      1. From back in the day before secularism got a stranglehold on Hoosier jurists comes this great excerpt via Indiana federal court judge Allan Sharp, dedicated to those many Indiana government attorneys (with whom I have dealt) who count the law as a mere tool, an optional tool that is not to be used when political correctness compels a more acceptable result than merely following the path that the law directs: ALLEN SHARP, District Judge. I. In a scene following a visit by Henry VIII to the home of Sir Thomas More, playwriter Robert Bolt puts the following words into the mouths of his characters: Margaret: Father, that man's bad. MORE: There is no law against that. ROPER: There is! God's law! MORE: Then God can arrest him. ROPER: Sophistication upon sophistication! MORE: No, sheer simplicity. The law, Roper, the law. I know what's legal not what's right. And I'll stick to what's legal. ROPER: Then you set man's law above God's! MORE: No, far below; but let me draw your attention to a fact I'm not God. The currents and eddies of right and wrong, which you find such plain sailing, I can't navigate. I'm no voyager. But in the thickets of law, oh, there I'm a forester. I doubt if there's a man alive who could follow me there, thank God... ALICE: (Exasperated, pointing after Rich) While you talk, he's gone! MORE: And go he should, if he was the Devil himself, until he broke the law! ROPER: So now you'd give the Devil benefit of law! MORE: Yes. What would you do? Cut a great road through the law to get after the Devil? ROPER: I'd cut down every law in England to do that! MORE: (Roused and excited) Oh? (Advances on Roper) And when the last law was down, and the Devil turned round on you where would you hide, Roper, the laws being flat? (He leaves *1257 him) This country's planted thick with laws from coast to coast man's laws, not God's and if you cut them down and you're just the man to do it d'you really think you would stand upright in the winds that would blow then? (Quietly) Yes, I'd give the Devil benefit of law, for my own safety's sake. ROPER: I have long suspected this; this is the golden calf; the law's your god. MORE: (Wearily) Oh, Roper, you're a fool, God's my god... (Rather bitterly) But I find him rather too (Very bitterly) subtle... I don't know where he is nor what he wants. ROPER: My God wants service, to the end and unremitting; nothing else! MORE: (Dryly) Are you sure that's God! He sounds like Moloch. But indeed it may be God And whoever hunts for me, Roper, God or Devil, will find me hiding in the thickets of the law! And I'll hide my daughter with me! Not hoist her up the mainmast of your seagoing principles! They put about too nimbly! (Exit More. They all look after him). Pgs. 65-67, A MAN FOR ALL SEASONS A Play in Two Acts, Robert Bolt, Random House, New York, 1960. Linley E. Pearson, Atty. Gen. of Indiana, Indianapolis, for defendants. Childs v. Duckworth, 509 F. Supp. 1254, 1256 (N.D. Ind. 1981) aff'd, 705 F.2d 915 (7th Cir. 1983)

      2. "Meanwhile small- and mid-size firms are getting squeezed and likely will not survive unless they become a boutique firm." I've been a business attorney in small, and now mid-size firm for over 30 years, and for over 30 years legal consultants have been preaching this exact same mantra of impending doom for small and mid-sized firms -- verbatim. This claim apparently helps them gin up merger opportunities from smaller firms who become convinced that they need to become larger overnight. The claim that large corporations are interested in cost-saving and efficiency has likewise been preached for decades, and is likewise bunk. If large corporations had any real interest in saving money they wouldn't use large law firms whose rates are substantially higher than those of high-quality mid-sized firms.

      3. The family is the foundation of all human government. That is the Grand Design. Modern governments throw off this Design and make bureaucratic war against the family, as does Hollywood and cultural elitists such as third wave feminists. Since WWII we have been on a ship of fools that way, with both the elite and government and their social engineering hacks relentlessly attacking the very foundation of social order. And their success? See it in the streets of Fergusson, on the food stamp doles (mostly broken families)and in the above article. Reject the Grand Design for true social function, enter the Glorious State to manage social dysfunction. Our Brave New World will be a prison camp, and we will welcome it as the only way to manage given the anarchy without it.

      4. When I hear 'Juvenile Lawyer' I think of an attorney helping a high school aged kid through the court system for a poor decision; like smashing mailboxes. Thank you for opening up my eyes to the bigger picture of the need for juvenile attorneys. It made me sad, but also fascinated, when it was explained, in the sixth paragraph, that parents making poor decisions (such as drug abuse) can cause situations where children need legal representation and aid from a lawyer.

      5. Some in the Hoosier legal elite consider this prayer recommended by the AG seditious, not to mention the Saint who pledged loyalty to God over King and went to the axe for so doing: "Thomas More, counselor of law and statesman of integrity, merry martyr and most human of saints: Pray that, for the glory of God and in the pursuit of His justice, I may be trustworthy with confidences, keen in study, accurate in analysis, correct in conclusion, able in argument, loyal to clients, honest with all, courteous to adversaries, ever attentive to conscience. Sit with me at my desk and listen with me to my clients' tales. Read with me in my library and stand always beside me so that today I shall not, to win a point, lose my soul. Pray that my family may find in me what yours found in you: friendship and courage, cheerfulness and charity, diligence in duties, counsel in adversity, patience in pain—their good servant, and God's first. Amen."

      ADVERTISEMENT