College of Foreign Languages Studies,Xinjiang University,Xinjiang,China Email:1492818555@qq.com

[Abstract]This study aims to compare the effects of using online Automated Writing Evaluation(AWE)and contentbased peer feedback using email on EFL Writing.An experimental study with two groups was conducted for 16 weeks among the first-year non-English majors to explore the impacts of two modes of internet-based feedback.The findings reveal that peer email feedback contributes more to students’motivation and engagement,compared with the AWE feedback,thus generating more significant impact on the development of college students’EFL writing performance.It is hoped that the study helps both teachers and students better understand the nature of writing and produces implications for the current Chinese EFL writing instruction.

[Keywords]Automated Writing Evaluation(AWE);peer email feedback;EFL writing

Introduction

EFL writing is playing a vital role in such an increasingly byte-sized world with the globalization background.Writing ability is clearly required and described in“China Standards of English”(2018).In ESL/EFL teaching and research,performance in writing has been a key indicator of learners’language proficiency.While EFL writing has become a headache for both teachers and students,and Chinese EFL writers are reported being weak in content organization(Wang,2005).

In the era of“Internet +”，the traditional college English writing teaching model cannot meet the requirements of contemporary college students any more．The Ministry of Education(2017)also emphasized the necessity of applying information technology in College English teaching.The epidemic COVID-19 makes online education a phenomenon not only in China but around the world since the early spring of 2020.Network-based writing teaching,then,is the“new normal”.For lack of time for writing class,feedback is crucial in improving non-English majors’EFL writing.Using internet-based feedback has been of a great help.

Literature Review

Feedback

As a key process in L2 writing and a central part of teachers’instructional repertories,feedback has taken on various forms over the past decades(Hyland & Hyland,2019; Lee,2017).From product-oriented feedback to content-focused process-oriented,from peer- feedback to self-evaluation(Lee,Mak & Yuan,2019).The studies yield inconsistent findings regarding the impact of computer-mediated communication(CMC)mode on the quality of peer feedback and text revisions(Yu&Lee,2016)in ESL/EFL writing contexts.The previous studies of feedback fall in 5 categories:(1)who responds? Is it peer response,teacher response or teacher-peer response?(2)How do they respond? Written feedback,face-to-face conferences,or some other format,e.g.using web.(3)What do they respond to? Content,form or both?(4)When do they respond? Between drafts or on the final draft?(5)Where do they respond?Between lines,in the margins or at the end?The last set of studies centers on investigating students’attitudes towards different types of response.The most controversial issue is the effectiveness of different types of response.

Automated Writing Evaluation(AWE)

Research followed in the Mid-90s when some typical AWE software were then applied in English writing teaching.Tang & Wu(2011)reviewed research on using online AWE in the classroom.Wang(2017)and Li & Zhong(2017)researched on the impacts of AWE on EFL students’writing proficiency.Most of the studies(e.g.Warschauer& Grimes，2008)focus on the application of AWE to the classroom teaching,including some empirical studies and the findings are positive in improving students’writing performance but with some challenges.

Comparative Studies on Different Modes of feedback

In the 1990s,feedback using email and BBS or Internet chat sites was popular.Most of the studies compared the effects of the two modes of feedback,either traditional or computer-mediated,focusing on students’writing apprehension,quality,and improvement or peer-response length,peer interaction types,and their influences on the revision.Ho and Savignon(2007)stressed the importance of using a combination of the two modes of peer feedback activities into classroom practice.ESL/EFL professionals in this field found no satisfying answer to the value of using the technology in the peer-feedback writing classes(Lee,2016).Little research has been done on comparing automated feedback and peer feedback using email through quantitate and qualitative studies.

Methodology

Research Context

As Epictetus put it,“if you wish to be a good writer,write.”Wang(2016)proposes the“Learn-Together-Use-Together”model after advocating write-to-learn with the L2-facilitating factors as its theoretical underpinning,i.e.communicative intention,interactive alignment and comprehension and production.Sato & McDonough(2019)finds that contextualized practice makes foreign language learners’spontaneous use more accurate.

This empirical study is to examine the different impacts of two modes of feedback by adopting a content-based peer email response in authentic writing and feedback using AWE system in two groups.It lasts a whole academic year.Tests,questionnaires,interviews and students’written drafts are used for data collection to respond to the following research questions:

1.Is peer email feedback more effective in EFL writing than automated feedback?

2.In which aspect is peer email feedback more effective in improving students’writing proficiency?

3.Which mode contributes more to students’motivation and engagement in EFL writing?

Design and Participants

The study involves 4 classes(taught by the researcher)from 4 different colleges,two classes serving as Group1— email feedback(N=83)and two as Group 2 — automated feedback(N=63).All the four classes complete a pre-test and a delayed post-test.Between the two tests,the participants in Group1 use AWE feedback while those in Group 2 finish their writing assignments by responding to emails with their pen pals.

The participants consisted of 146 students from different majors(male 94,female 52)They were all Grade 2019 undergraduates and of engineering background.They were between 18 and 20 years old and were from different cities,towns or villages around China.They were all in their first year of taking college English course.

Pilot Study and Instrument

Prior to the real study,a pilot study was conducted in the first semester of 2019-2020 Academic year to examine the validity and reliability of the test instrument and capture the tendency of the program’s effects on students’EFL writing.

A writing test was written based on Liu & Han’s study(1999),with main discourse features required in a text with overall quality,comprising 7 large items under 3 parts.The Cronbach alpha is reported as 0.895 in the post-test after adjustment in difficulty of some items,numbers or scores.The Rating scale is developed based on language testing theory and Chiang’s(2003)study to assess students’written work and reports a=0.913.The favorable results tell that the post-test paper in the pilot study can be used as the test instrument for the real empirical research.

The questionnaire was developed based on Yoshiyuki’s(2006)study investigate intrinsic motivation(8 items),beliefs(6 items),efforts(3 items),strategies(2 items)，anxiety(1 item)and extrinsic motivation(1item).This is of a 5-point Linkert scale ranging from completely disagreement(1point)to completely agreement(5points).The higher score the students get,the higher the level their motivation and engagement.After removing those right-skewed items(ceiling effect revealed by the mean and standard deviation),a Cronbach alpha to check the internal consistency was computed for the 10 items and a satisfactory score of 0.911 was obtained,which means the reliability of the questionnaire instrument is ensured.

Treatment and Data Collection

Treatment for Group 1.

1.The researcher and students register in an AWE network.

2.The researcher uploads the requirements of the composition needed to be written by students every two weeks.

3.The students write compositions according to the requirements.Then they submit the compositions within the required time.

4.The system offers students holistic score,general comment and diagnostic feedback immediately on spelling,grammar,word usage or collocation existing in the sentence.

5.The students can correct their compositions repetitively within the required time.

6.The researcher comments on students’writing on some typical problems in class.

Treatment for Group 2.

1.The researcher match the students in two classes randomly as pen pals who exchange their own email box and write a letter introducing themselves and anything they like to write.

2.Pen pals exchange emails freely with a writing task as the main content every two weeks.

3.From Week 1 to Week 4,the researcher doesn’t read students’letters and let them know it.

4.From Week 5 to Week 8,the researcher reads some emails without correcting errors,gives a general comment on the content and emphasizes the coherence in their letters.

5.From Week 9 to Week 13,the researcher underlines impressive sentences in the emails and explains typical linguistic errors in class,leaving them corrected by students themselves.

6.From Week 14 to Week 16,the students read their satisfying sample letters in class.

Data Collection.

Each participant took the pre-test before the experiment and the post-test after 16 weeks’program.The test scores were anonymously obtained by two English teachers rating the composition(one native English speaker and the other non-native speaker).the Person Product Correlation is r=95.1 for the two sets of scores,and the average scores are taken as the final ones.In addition,documents of students’written drafts and 135 valid questionnaires(60 from Group1and 75 from Group 2)were collected to analyze the effects of the different modes of response on students’writing performance and writing motivation.An interview was conducted among 4 students randomly chosen from either group.

Results and Discussion

Effects on Writing Proficiency

The results of Independent Samples Tests show that there is no significant difference between the two groups before and after the intervention in the pilot study with M=59.000(S.D.= 8.109)in the post-test of Group1，M=60.135(S.D.= 8.797)in Group 2,and P=0.546>0.05.It indicates that one semester’s training does not work out significant difference in students’writing in both groups.But,in the post-test of the real study in the second semester,the results display distinct difference with M=60.538(S.D.=8.863)in Group1,M=68.157(S.D.=10.175)in Group 2 and P=0.006>0.05.Thus，we may accept peers email feedback is more effective than the automated feedback.

The descriptive statistics of the three parts of the post-test scores for two groups reveal no significant effect on morphology but more positive effect on coherence(p=0.010461＜0.05)and syntax(p=0.026901＜0.05)in Group 2.In other words,content-focused email response is more effective in improving coherence,and at the same time does help to improve syntax.And Table 1 shows significant differences in both coherence and syntax at 0.000 level and 0.002 level，i.e.the students in Group 2 improve more significantly in constructing a coherent text than in other aspects.

Table 1.Comparison in three parts of the writing tests in Group 2

Some sample writings of different levels(good and poor)are randomly selected respectively from 2 groups.It’s found that students in Group 2 have a better control in introducing the topic,elaborating and supporting the view,logic and transition and using cohesive structure.This result echoes previous studies(e.g.Jones et al.,2006)on the effects of CMC peer feedback,i.e.providing opportunities for meaning negotiation and interaction in a less threatening environment,and enable students to offer feedback focusing on content and text structure.

Effects on Writing Motivation and Engagement

The data in Independent Samples Test of questionnaires shows that three items are significantly different between two groups at 0.000 level(intrinsic motivation),at 0.046 level(language use anxiety)and at 0.001 level(language leaning belief).It implies that after 16 weeks’intervention,more students in Group 2 hold positive attitude towards English writing and students gain more confidence in English writing than those with automated feedback.More appreciable answers in Item 5 indicate more awareness in self-editing in peer email response; most students from both groups don’t like finishing writing composition tasks;some in Group 2 are not satisfied,for they have doubt about whether they can achieve accuracy without teacher’s corrective feedback.This may partly explain why it received a lower mean score than Group1 in Item 9.

Data also reveals that the students’writing frequency(2.8/per week)and average length of their written work(290 words)in Group 2 nearly doubled those in Group 1(1.5/per week and about 150 words).Most of them(58.15%)in Group1 write to finish homework,while nearly half of the students（47.29%）in Group 2 write out of their own will.Furthermore,over 80% of the students in both groups choose emails or journals as their favorite writing styles because they can write whatever they like and especially express their understanding of life freely.

According to the interview about students’comments on feedback,AWE feedback is quick but general，more on grammar and vocabulary but less on the organization and content，and students often feel puzzled how to revise the writing even with the suggestion.They sometimes doubt the bugs in machine and hesitate to respond and prefer teacher’s feedback.Students in Group 2 found they were curious and even excited when receiving peer’s email and can’t wait to respond immediately,agree,disagree or debate and they feel free to express themselves and at ease to write more and fluently with new ideas.It’s time-consuming but relaxing.

Conclusion

The findings of the experimental study in EFL settings demonstrate positive effects of peer email-exchange activities on their writing performance and more improvement in coherence and organization.The results of the investigation show that students are more motivated and engaged in peer email feedback than those in automated feedback.The task of writing emails of honest praise or criticism hoping for a response often increases students’desire to convey their meanings,thus realizing a real communication through writing.This echoes previous studies(Belisle,1996;Gu,1998)on using email in English writing that students are more likely to have new ideas and complete a qualified essay through collaborative questioning,suggesting,commenting and editing.And it may touch a fundamental issue that faces L2 teaching(Wang,2020)on how to help learners get over learning difficulty,i.e.appropriately mapping language onto fluid dynamic ideas.In order to discover whether the improvement of students’writing proficiency follows the increased writing quantity with longer intervention and what detailed changes occur in peers’negotiations，longitudinal studies and formative assessment are required.

Acknowledgments

The author is a part-time researcher at Xinjiang Center for Comparative Studies of Chinese and Foreign Cultures&Intercultural Communication Research.

The research is funded by College of Foreign Languages in Xinjiang University(WY20161208，WY20171202,WY20191203),Xinjiang University（50012000108）and by Xinjiang Center for Comparative Studies of Chinese and Foreign Cultures&Intercultural Communication Research(010716C03).