Tuesday, November 12, 2013

State Tests and Dual enrollment: From one who has seen

The joy of homeschooling in its purest form, is that you do not have to bend to anyone’s core or curriculum. And you certainly don’t have to take any state mandated tests here in Utah as of yet. As a parent, you can control what you expose your children to at some degree. You set the scope, the sequence and the assessment.

Someone asked why I would want to be involved with the new state assessment process. To be honest, I have asked myself this very question. Truth be known, when my children were younger we were home schoolers in the purest sense of the word. We had so much fun traveling and experiencing learning. As they grew, I needed additional resources to make sure they could reach their potential and thankfully the Utah system provided some of those resources.

I could not provide a chemistry lab, the upper maths, or the associates program cost effeciently for my older child in a timely manner and meet the needs of my younger children. Finding private tutors was extremely difficult and the public school system already had the courses with some great educators. I am what some parents would consider a dual enrollment parent. I am very eclectic in my approach to homeschool and would consider myself an opportunist seeking the right mentors for my children. As a result, my children will be subject to the state testing in one way or another.

With the new FERPA laws and the ability to track data, I feel it imperative that core knowledge is tested without the social agendas. A proper test would include facts and not any of the controversial, subjective, social, or the psychosomatic testing. If USOE or our school system is to be trusted, it is imperative to remove all items related to any social agenda or touchy feely stuff and stick to factual knowledge or proven theories. We have enough factual knowledge; we don't need any of the extra stuff.

15 parents were chosen to review the new Utah common Core high stake tests. Those 15 parents selected by USOE, the Senate, and the House of Representatives will have the opportunity to serve on this panel for the next two years and others will serve four. I have chosen to serve four years. This group of individuals are a thoughtful, concerned, and a strong bunch.

The test (for all grade levels) was divided amongst the 15 of us. Each 1/15th was divided into 4 batches consisting roughly of 660 questions. I personally reviewed 9 batches. I scored the questions in the following manner:

•Those I felt were too subjective and inappropriate for a testing situation, I marked no and left an explanation in the comment box.

•Those I felt were appropriate questions, but needed some revision, editing or review of the stimuli,  I marked yes but left comments in the box of possible solutions or options that could be delivered.

•Those I felt were appropriate for a test for example math, I marked yes.

Each question was reviewed twice and some reviewed three times by different parents. Out of the 10,000 questions reviewed, only approximately 600 had concerns. Relatively a small number. Those concerns ranged from content, grammar, functionality and more. In the 9 batches I was able to review, I only saw two questions that really rubbed me wrong. I presented my concerns to the group and we discussed better solutions and helpful comments.

We were asked to sign a confidentiality agreement where we could discuss the test but not the core content questions outside of our time there. So you will not find any content of the test here. If it is in the core then it is probably on the test.

What did I learn?

First, if I had a concern, "write it in the comments box". The people monitoring would not answer questions but would facilitate our ability to comment on what we saw. They were non-partial and very attentive to our needs. A pleasant and wonderful staff. It was a pleasure to work with them and those on the parent panel.

After finding out the vender for the test would be AIR, I was very concerned since AIR's forte is psychological testing. Having seen psychological tests, I walked in thinking I would see the situational or touchy-feely questions. I did not see any such questions. I was pleased to see so many uplifting, informative and local referenced passages. Any passages which appeared to have subjective text, answers that led children to believe an opinion as fact or social agendas were addressed not only by myself but by many of the parents there. Each question was reviewed by two or three parents.

As a former 3-5th grade Spanish Immersion Teacher and Science Specialist, it became very apparent that science and math teachers will need to do a thorough job of covering the core. Also, It was apparent that parents need to take advantage of the sage parent practice tests so that the children are familiar with the new formatting and test taking procedures. It is not just multiple choice. There are drag and drop, highlighting, text responses, and interactive questions in all the areas. Children will need to learn to type before 3rd grade for the writing evaluations.

There is a help portal for parents here:
http://www.sageportal.com

There will be Training Tests supposedly starting in December which will not have questions from the core but rather a sampling of the prototypes of questions to be used to get answers. Student must learn how to manipulate the mouse, drag, drop and perform other needed computer skills. The typing skills including sentence structure, capitalization and such will need to be addressed much sooner including how to use a mouse and not a touch screen.

What is tested?

Reading, Math, Science, Writing – If it is in the core, then it is covered. As for specifics, I will not address that ever.

Reading/Language Arts were much the same as before. I was surprised there wasn’t a reading level diagnostic but I am assuming the schools will already have that implemented somewhere else. The subject matter comes from all over but what I saw was uplifting. Those I vetted were the ones best left in a classroom discussion situation rather than a testing situation.

Math will no longer have just multiple-choice answers to choose from but rather now they will type in the answer. The ability to solve the problem correctly will be vital. Students will need to know the process and be able to type in the answer to the question. No longer will they be able to figure out the questions by utilizing the answers given. There are fewer multiple choice answers, you have to use the numbers given and type your answer in. 3-6th will not have a calculator and there are no fuzzy answers. Part of the 6th grade test and grades 7-11 will be allowed access to a calculator either through the software, or they are allowed to bring their own approved calculator.

I think Science was my favorite part. In science, they will need to understand the subject matter and how to perform an experiment. There will no longer be any guessing for they will be required to make conclusions from the data they create while utilizing the scientific method. Biology better know biology, chemistry better know chemistry. For those who have been taught well, this test will actually be fun. There will be drag and drops and everything you can imagine. They will need to understand vocabulary and models not just definitions (burn the crosswords!) as well as feel comfortable using a mouse.

Writing will be much easier since there is more than a prompt. There is reading text to draw conclusions from. They will need to utilize the information given and write a composition supporting the prompt using the information provided. The students will have access to spelling check but grammar check will not be allowed. The student will have supporting as well as opposing text that will help the student create their own ideas and formulate their own conclusions. The topics are across the board. The structure of persuasive essay and informational essays will be key. You can listen to the teacher webinar training here: http://connect.schools.utah.gov/p141zhflhsf/

Concerns I have for the test

Since this is history in the making, where parents vet the test, I would hope that the parent panel could be involved sooner so that the subjective questions could be eliminated prior to the technological implementation. It would be more cost effective for everyone involved. There were a few reading texts and science questions that need to be vetted but overall there were far less objective questions than I presupposed going into the evaluation. And it was nice to see that as a parent, I was not the only parent who was concerned regarding the few questions that were subjective and leaned towards opinion verses factual.

I am concerned that schools will be graded upon this test when it still has yet to be piloted. Through the pilot they will be able to gather the necessary statistics to provide a proper mean, and medium for grading. Until the test has enough test questions in its bank, the questions vetted for the children's ability to understand and complete, and the rubrics are properly establish for the writing, there is no fair or proper way to establish a grading system for those who are required to take the test.

I am confident and certain that homeschoolers should not be required to take this test EVER. Most don't teach to this exam. Many of us follow a traditional math sequence so the grade levels may not coincide. And most do not focus upon teaching their children typing skills in the third grade.

I am concerned that we as the parent panel get to know of the modifications and resolutions for the questions for which we were concerned. We are told they will be addressed but at this point we may not get to see the changes until we review the next set of questions. Out of the 10,000 questions there were only a small percentage of questions that need to be addressed. I would hope in some time in the future we could find out if they were addressed or not. By removing any subjective questions and utilizing only facts for the test, I feel that this test can be a fair method to judge learning and growth but only after it is vetted and piloted correctly.

Ways to prepare a child

Remember if you are going to utilize funds or resources distributed through the government, there will always be strings attached. If you want to know which tests will be required of your student go to here:

Let your child have a blog. Keep it private but have them write their responses to the reading, science and anything that could possibly be a writing response. May even use the blog for a nature journal and print the information and paste in their nature journals.

Don't get caught up in fuzzy math. Fluency in math facts are imperative not only for memory but for problem solving quickly and effectively. They don't have fuzzy explanations on the test.

Use a mouse. The touch screen is wonderful! But these tests are not created for a touch screen. (Maybe in the future since iPads seems to be the vital classroom tool.)

Use the practice tests on the new sage site when available.

And finally…yes, you can "opt-out." The problem with opting out is this: Current legislation has it that if you opt out your child, your child is seen as "non-proficient." So "opt-out" carries the stigma of "non-proficient." If a school has a large population that is determined non proficient, then the lower the score for the school grade in turn can effect the school negatively. Currently 5 percent of the schools population can (opt-out/refuse to take the test) without any negative consequences.  How does this effect those who utilize charter schools or alternative online programs?

First,  the schools from which their SIS funding comes from, will only want a few of their students opting-out.  For a charter school with 78 students, three students can opt out.  For a school with 300 hundred students, they can have 15 students refuse to take the test.  This means a form must be signed and an "official opt out" is created.  A student who doesn't show up for testing receives even a worse score. Thus, it is imperative to make sure if you are opting out that you notify your school immediately.  You always take the chance of the school dropping you when notifying them but, even worse yet, a no show has a trickle down effect. The schools will begin to refuse to offer the optional programs to homeschoolers. The charter schools are catching on quickly and dropping the programs of Harmony or My Tech High.  Thus you find mandatory testing requirements for those programs.

They are giving a nine week window for the schools to offer the tests so there is relatively few reasons why a no show occurs. Also this test is not timed.  A child could stop and return back to the test.  The time allotment will be determined by the schools.

Overall, this parent panel and review was an amazing experience. I imagine that if this process was implemented in the former state tests there would be far more trust from the public. If the comments from the panel are implemented as we are told, then I would feel confident that my son could take this test without any negative repercussions. And in four years, when the test is refined, it could be used to evaluate teachers for content and schools for instruction. Although let me make it clear, I don't see assessment as a solution. It is a tool. But that is topic for another blog someday.

UPDATE 11/22/13
SINCE WRITING THIS I RECEIVED THE FOLLOWING:
Dear Parent Reviewers,

We would like to thank you for the tremendous work you did in reviewing all 10,000 SAGE Summative test items. Outlined  below is the process which USOE will implement to respond to the feedback provided by the parent item review committee:

1.    USOE content staff and AIR content staff will independently review all item feedback and propose resolutions.
Resolutions may include the following: removing item, editing item, and flagging item for further review. After testing all items will have an extensive statistical review including those flagged by the review committee

2.    USOE content staff and AIR content staff will work collaboratively to determine proposed resolution for all item feedback.  USOE will make final determination for any issues of disagreement

3.    Proposed resolutions will be reviewed by content/bias review committees

4.    Final resolutions will be implemented

5.    Parent Review Panel will be provided with summary statistics of item resolutions (number of deleted, edited, etc.)

6.    Parent Review Panel will review all items which were edited (Fall 2014)

7.    Parent Review Panel will review all new items written for 2015 administration (Fall 2014)



John Jesse
Director of Assessment and Accountability
Utah State Office of Education

UPDATE  1/14/14
Dear Parent Review Committee Member,

We want to thank each of you again for your tremendous work in reviewing all the items in the SAGE Summative item bank prior to students responding to them during the operational field test this spring. Your feedback was a critical component in moving this complex project forward.

USOE has now completed the initial review and response to the feedback you provided.  The process included the individual review of each assessment item that contained a comment from the parent review panel by individual USOE content experts.  Items were then either removed from the item bank, edited based on parent and review panel recommendations or flagged for further review after the item is field tested. 

All non-removed items were brought before an item review committee consisting of regular education teachers, content experts, special education teachers, and minority group representatives for reevaluation.  All parent concerns were addressed by checking functionality, bias, style, facts, and content on all flagged items.

The majority of flagged items were targeted for further review after field testing. For example, the identified issue for an item by the parent review panel may have been the item was too difficult for the grade level. The item was re-checked by content experts and deemed grade appropriate but will be reviewed again after field testing by both content experts and the parent review panel. All items flagged by the parent review panel, which were not removed will be presented to the parent review panel next summer for further review.

Listed below is the numeric representation of the process:


Flagged
Removed
Changed
Review after Field Test
English Language Arts
297
27
97
173
Mathematics
71
4
4
63
Science
229
12
56
161

John Jesse
Director of Assessment and Accountability
Utah State Office of Education

9 comments:

  1. Thank you very much for this informative blog post. As a homeschool parent who has children in My Tech, I really appreciate this information.

    ReplyDelete
  2. Thank you, thank you for sharing your experience. It has helped me make a more informed decision about the course of our family's schooling experience.

    ReplyDelete
  3. Thank you Kim for your informative review. I have heard a lot of negative hype about the SAGE testing. I hope that the questions are not something I should be concerned about.

    I did have one question. You said that their are not time limits and I have been told the same by Mytechhigh however as I was researching the tests I found the following on the SAGE resource site:

    If students have not completed the SAGE Writing assessment after 130 minutes of total test time, it is appropriate to announce the following: “Students it is now time to make your concluding statements and finish your essays. Please complete your essays in the next few minutes.“
    http://sage.jordandistrict.org/

    So would there be no limit as long as they finish it within the allotted time?

    ReplyDelete
    Replies
    1. At this point, being a very conservative mom, I am not concerned about the questions. But I know things can change, that is why I signed up for the whole possible term.

      We are told over and over at the USOE that there is no time limit. But it appears that the districts are choosing that for themselves. It appears it will depend upon where you are taking the test.

      At one school, the teachers allowed the students all day but they were not allowed to go to lunch with their peers to discuss the test. Lunch was brought to them. I was told it was ok for the students to go to lunch or even home and come back. They are looking for a writing sample which the student must do on his own. The writing sample must include a thesis, three supporting ideas and a conclusion. But the validity of the ideas can come from the student, the reading articles and even home if needed. This is the first year so I am watching to see if this really comes to pass. It will vary from district to district because they are allowed to interpret the rules. This is local control.

      If concerned about time, be sure and make sure your child is given the time they need without the pressure by speaking with those in charge. Let them know you are aware that USOE as given no time limit and you are willing to bring in lunch and make sure your child has the time to complete the test they need.

      Delete
    2. Here is my direct reply for Judy Park regarding the time limits. They are worried about testing fatigue.

      On Mar 27, 2014, at 10:28 AM, "Park, Judy" wrote:

      Although the test is not timed, we have received reports of students taking as long as 5 hours. This is not advisable for multiple reasons the most important being student test fatigue. Good writing is not the same as lengthy responses. To protect students from having exhausting testing experiences and tying up classrooms and technology unnecessarily we have provided the guidance you cite which prevents testing from being longer than wise. I hope this clarification is helpful.
      Thanks
      Judy

      Delete
  4. I very much appreciate your information, Kim. My concern in your entire post was that students need to know how to type before 3rd grade! What?! I think learning to type before 3rd grade is nonsense.

    ReplyDelete
  5. I'm not certain how you get that the entire post was about typing before third grade. It will be a necessary skill for success on the SAGE starting in 3rd grade but I never said it was a necessary skill for life. If you click on the other tabs of this blog you would see that I don't put my children in the public system before 6th grade because I use an entirely different system to teach my children to write and they have done very well for themselves. These are just suggestions for parents who have subjected their children to the test and possible ideas to alleviate their concerns.

    ReplyDelete
  6. Oh Kim! Your post didn't come across that way at all! I was simply stating (and obviously not very well) that this seems like so much pressure as it is and then add onto that needing to type by age 8. To what extent they need to type...I don't know. I am very grateful to you for sharing your experience as testing will be something that will come into play for us starting next year. This is very helpful in determining whether or not we want to enroll with a charter next year. I have poked around your blog a bit and always read your remarks on WHEN and I appreciate the advice and support you offer our community!

    ReplyDelete
  7. Thank you for posting this. It has been very informative. It does however bring up several concerns. With the amount of flagged material so many were changed and then will not be reviewed by the panel again before our children are tested.(Did I understand that correctly?) Also the amount that was flagged that were not removed or changed. With such a small group of people reviewing such a large amount of data, especially in comparison to the amount of students and parents throughout the state, would there be a chance that you would miss something. The test is progressive , so while one question on its own or out of sequence can seem harmless and less "behavior" or politically charged, but given the right sequence could the questions mine for behavioral or implicate a political agenda. You mentioned the Ferpa law change, it is my understanding that one of th changes made without congressional approval was something to the effect of those who are "associated" with the education system including "volunteers" will have access to information such as this data. Does this mean AIR will have access to our childrens data, because they created the test and are now associated with the education system. Also I know you did not touch on this, but in my research regarding Common Core and the SAGE testing, I came across the Affordable healthcare act, in one of the tiers of the act not yet in play, it goes into the government having access to such data, and using it to determine mental health of the well being of its citizens etc. How will the combination of these laws andstandars affect the future of our children?
    I guess another concern is no one has actually seen the test in the way it is being administered to our children. Another concern is that the score for each child isnt given, though with the new FERPA laws I suppose we can jump through the hoops to get the scores. After so many hours of testing dont our children desreve to know how they did?

    Also with SB0122 being signed this last week schools and teachers will no longer be affected by those students that opt out of the test.

    Sorry for so many concerns but if you have any answers you can give, I know you had to sign a confidentality clause to view the test. I think every parent should be afforded that same opportunity.

    Also, what homeschool are you teaching through it has been from what I have read if you go through a school that accepted money from the RTT grant they must follow the Common Core standars including the admin of the Sage test.

    ReplyDelete

Please leave your name since I love to get to know everybody!