Our school had the same problem in the beginning using Running Records. We found that scores would vary occasionally for many reasons: fidelity, time of day, who implemented, and even whether or not the student had breakfast that morning.

The key our success came when we recorded more data points. One or two "off" scores will not affect your decisions if you are working with many data points. Take scores more often and record them on a simple graph. The trend will be obvious.

It is not unusual for us to have 10 scores recorded in two weeks. (one per day) This makes the outliers very obvious.

This is true for whatever assessment you are using.

Hope this helps!

Quote is taken from Stanley Deno's PowerPoint Presentation: Leadership for Developing a School-wide Progress Monitoring System.

Running records and informal reading inventories (IRIs) focus on what might be taught in an effort to improve reading; whereas, CBMs are outcome indicators that reflect on the success of what is taught. A large body of research has shown that one-minute samples of the number of words read correctly from reading passages are sensitive, reliable, and valid of measures of reading growth. If teachers find them useful, running records and IRI’s can be used in conjunction with regular progress monitoring to help generate ideas for possible changes in students’ programs that can be evaluated using CBM.

FYI