Traditional assessment does not confirm and result in learning. Assessment in Malaysia is now shifting from assessment of learning and for learning to assessment as learning to prepare students to face life beyond classroom to meet global challenges. This study intends to identify the challenges and needs of ESL teachers in implementing alternative assessment in the form of portfolio assessment in teaching English language. Qualitative study specifically online classroom observation and interviews with five ESL teachers were carried in the district of Perak. Findings revealed that the teachers still faced challenges in implementing alternative assessment in teaching English language. These challenges include selecting the content and learning standards to determine assessment activities, selection of authentic assessment, assessing student work based on learning objectives, providing feedback and educating students on using feedback to improve learning. Finally, the findings on teachers' need for portfolio assessment showed that teachers preferred a range of authentic assessments with proper instructions and rubrics to guide them. Teachers expressed the need for training to expose them on the 'what' and 'how' to implement alternative assessment in classrooms.